In adult tissues such as adipose tissue, post-mitotic cells like adipocytes can be replaced by differentiation of a population of tissue-resident stem cells. Expression of mouse telomerase reverse transcriptase (mTert) is a hallmark of stem cell populations, and previous efforts to identify tissue-resident adult stem cells by measuring mTert expression have increased our understanding of stem cell biology significantly. Here, we used a doxycycline-inducible mouse model to perform longitudinal, live-animal lineage-tracing of mTert-expressing cells for more than 1 year. We identified a rare (<2%) population of stem cells in different fat depots that express putative preadipocyte markers. The adipose-derived mTert-positive cells are capable of self-renewal and possess adipogenic potential. Finally, we demonstrate that high-fat diet (HFD) can initiate differentiation of these cells in vivo. These data identify a population of adipose stem cells that contribute to the depot-specific response to HFD.