Topologically associating domains (TADs) are thought to play an important role in preventing gene misexpression by spatially constraining enhancer-promoter contacts. The deleterious nature of gene misexpression implies that TADs should, therefore, be conserved among related species. Several early studies comparing chromosome conformation between species reported high levels of TAD conservation; however, more recent studies have questioned these results. Furthermore, recent work suggests that TAD reorganization is not associated with extensive changes in gene expression. Here, we investigate the evolutionary conservation of TADs among 11 species of Drosophila. We use Hi-C data to identify TADs in each species and employ a comparative phylogenetic approach to derive empirical estimates of the rate of TAD evolution. Surprisingly, we find that TADs evolve rapidly. However, we also find that the rate of evolution depends on the chromatin state of the TAD, with TADs enriched for developmentally regulated chromatin evolving significantly slower than TADs enriched for broadly expressed, active chromatin. We also find that, after controlling for differences in chromatin state, highly conserved TADs do not exhibit higher levels of gene expression constraint. These results suggest that, in general, most TADs evolve rapidly and their divergence is not associated with widespread changes in gene expression. However, higher levels of evolutionary conservation and gene expression constraints in TADs enriched for developmentally regulated chromatin suggest that these TAD subtypes may be more important for regulating gene expression, likely due to the larger number of long-distance enhancer-promoter contacts associated with developmental genes.