The translocator protein TSPO is in an important diagnostic and therapeutic target in a range of pathologies, including neuroinflammation and cancer. Despite the availability of several structures of TSPO homologues, our understanding of the molecular determinants that govern high-affinity interactions of TSPO with its ligands is incomplete. Here, in order to decipher the key structural elements of TSPO responsible for interactions with its ligands, we designed a panel of chimeric proteins mimicking the mammalian substrate binding site grafted onto the backbone of the Rhodobacter sphaeroides TSPO homologue, RsTSPO. One of the designed chimeric constructs, RsMouse, could be heterologously expressed and displayed improved binding affinities for the known TSPO drugs diazepam, PK11195 and NBD-FGIN-1-27. Furthermore, the chimeric protein had improved interactions with NBD-cholesterol, a fluorescent analogue of the presumed natural substrate of TSPO. Partial modifications of the transmembrane helix bundle in the chimeric construct differentially affected binding of the TSPO drugs and the natural substrates of TSPO, consistent with the presence of multiple ligand binding sites in the protein. Based on the available structures of TSPO homologues, the substrate interactions may involve a lateral opening of the protein in the TM1-3, and stabilisation of TM4-5 is important for drug-like ligand binding. These observations are consistent with our experimental results, which show that the determinants of high-affinity ligand interactions of TSPO are distinct for different classes of ligands.