Bacterial populations in clinical and laboratory settings contain a significant proportion of mutants with elevated mutation rates (mutators). Mutators have a particular advantage when multiple beneficial mutations are needed for fitness, as in antibiotic resistance. Nevertheless, high mutation rates potentially lead to increasing numbers of deleterious mutations and subsequently to the decreased fitness of mutators. To test how fitness changed with mutation accumulation, genome sequencing and fitness assays of nine Escherichia coli mutY mutators were undertaken in an evolving chemostat population at three time points. Unexpectedly, the fitness in members of the mutator subpopulation became constant despite a growing number of mutations over time. To test if the accumulated mutations affected fitness, we replaced each of the known beneficial mutations with wild-type alleles in a mutator isolate. We found that the other 25 accumulated mutations were not deleterious. Our results suggest that isolates with deleterious mutations are eliminated by competition in a continuous culture, leaving mutators with mostly neutral mutations. Interestingly, the mutator-non-mutator balance in the population reversed after the fitness plateau of mutators was reached, suggesting that the mutator-non-mutator ratio in populations has more to do with competition between members of the population than the accumulation of deleterious mutations.