Thermodynamic stability, in particular, the positivity of the specific heat in the microcanonical ensemble, is not an automatic consequence of the thermodynamic limit. But it holds under special circumstances such as for the most important case of quantum-mechanical Coulomb systems. Therefore, it is surprising that there are experimental indications to the contrary. In this Letter we study a simple model for which the microcanonical specific heat is positive, if the system is ergodic. However, if the system is not ergodic, the energy shell in phase space has some ergodic components with a negative specific heat. This provides another possible general pathway for a negative specific heat in addition to the commonly accepted, the small number of particles.