When stress in concrete exceeds certain threshold value, microcracks are nucleated, these microcracks can propagate and coalesce forming macrocracks, resulting in the gradual decay of the mechanical properties of concrete and eventual failure of the concrete structures. For safety concerns, one needs to develop suitable nondestructive testing methods capable of detecting past overloads of concrete structures during its service life. In this work, the stress-induced damage in concrete is monitored using ultrasonic techniques, exploiting the coupling between the stress level experienced by concrete and its wave propagation parameters. Cyclic compression tests with increasing maximum load level have been performed on specimens made of concrete with coarse-grained (CG) aggregates. Experimental results have been analyzed by two different ultrasonic methods—the linear and the nonlinear ultrasonic techniques. In linear ultrasonic technique, the stress level experienced by the specimens is related to the variations in signal amplitude and velocity of ultrasonic waves. In nonlinear ultrasonic method, the sideband peak count (SPC) technique is used for revealing the stress-induced damage corresponding to each load step. In comparison to linear ultrasonic parameters, the nonlinear ultrasonic parameter SPC-I appears to be more sensitive to the variations of the internal material structures during both loading and unloading phases. Moreover, the SPC technique has shown to be capable of identifying both the initial damage due to the evolution and nucleation of microcracks at the microscopic scale, and the subsequent damages induced by high overload, resulting in an irreversible degradation of the mechanical properties.