We study how inertia affects the behavior of self-propelled particles moving through a viscous solvent by employing the underdamped version of the active Ornstein-Uhlenbeck model. We consider both potential-free and harmonically confined underdamped active particles and investigate how the single-particle trajectories change as the drag coefficient is varied. In both cases, we obtain the matrix of correlations between the position, velocity, and self-propulsion and the explicit form of the steady-state probability distribution function. Our results reveal the existence of marked equal-time correlations between velocity and active force in the non-equilibrium steady state. Inertia also affects the time-dependent properties of the active particles and leads to non-monotonic decay of the two-time correlation functions of particle positions and velocities. We also study how the virial pressure of particles confined to harmonic traps changes as one goes from the overdamped to the underdamped regime. Finally, the study of the correlations in the underdamped regime is extended to the case of a chain of active particles interacting via harmonic springs.