Availability of the three-dimensional structure of the trp repressor of Escherichia coli and a large group of repressor mutants has permitted the identification and analysis of mutants with substitutions of the amino acid residues that form the tryptophan binding pocket. Mutant aporepressors selected for study were overproduced using a multicopy expression plasmid. Equilibrium dialysis with 14C-tryptophan and purified mutant and wild type aporepressors was employed to determine tryptophan binding constants. The results obtained indicate that replacement of threonine 44 by methionine (TM44) or arginine 84 by histidine (RH84) lowers the affinity for tryptophan approximately two- and four-fold, respectively. Replacement of arginine 54 by histidine (RH84) or glycine 85 by arginine (GR85) results in complete loss of tryptophan binding activity. Purified mutant and wild type aporepressors were used in in vitro heterodimer studies. The trp repressor of E. coli functions as a stable dimer. A large number of trp repressor mutants produces defective repressors that are transdominant to the wild type repressor in vivo. The transdominance presumably results from the formation of inactive or slightly active heterodimers between the mutant and wild type polypeptide subunits. An in vitro assay was developed to detect and measure heterodimer formation. Heterodimer formation was thermally induced, and heterodimers were separated on nondenaturing polyacrylamide gels. Aporepressors readily formed heterodimers upon treatment at 65 degrees C for 3 minutes. Heterodimer formation was significantly retarded by the presence of the corepressor, L-tryptophan. Indole-3-propionic acid, 5-methyl tryptophan, and other analogs of tryptophan, as well as indole, also inhibited heterodimer formation.(ABSTRACT TRUNCATED AT 250 WORDS)