The next generation of force fields for molecular dynamics will be developed using a wealth of data. Training systematically with experimental data remains a challenge, however, especially for machine-learning potentials. Differentiable molecular simulation calculates gradients of observables with respect to parameters through molecular dynamics trajectories. Here, we improve this approach by explicitly calculating gradients using a reverse-time simulation with effectively constant memory cost and a computation count similar to the forward simulation. The method is applied to learn all-atom water and gas diffusion models with different functional forms and to train a machine-learning potential for diamond from scratch. Comparison to ensemble reweighting indicates that reversible simulation can provide more accurate gradients and train to match time-dependent observables.