The interaction of a radiolabeled muscarinic cholinergic receptor agonist, [methyl-3H]oxotremorine acetate [( 3H]OXO), with a washed membrane preparation derived from rat heart, has been studied. In binding assays at 4 degrees C, the rate constants for association and dissociation of [3H]OXO were 2 X 10(7) M-1 min-1 and 5 X 10(-3) min-1, respectively, Saturation binding isotherms indicated that binding was to a single population of sites with a Kd of approximately 300 pM. The density of [3H]OXO binding sites (90-100 fmol/mg of protein) was approximately 75% of that determined for the radiolabeled receptor antagonist [3H]quinuclidinyl benzilate. Both muscarinic receptor agonists and antagonists inhibited the binding of [3H]OXO with high affinity and Hill slopes of approximately one. Guanine nucleotides completely inhibited the binding of [3H]OXO. This effect was on the maximum binding (Bmax) of [3H]OXO with no change occurring in the Kd; the order of potency for five nucleotides was guanosine 5'-O-(3-thio-triphosphate) greater than 5'-guanylylimidodiphosphate greater than GTP greater than or equal to guanosine/diphosphate greater than GMP. The [3H]OXO-induced interaction of muscarinic receptors with a guanine nucleotide binding protein was stable to solubilization. That is, membrane receptors that were prelabeled with [3H]OXO could be solubilized with digitonin, and the addition of guanine nucleotides to the soluble, [3H]OXO-labeled complex resulted in dissociation of [3H]OXO from the receptor. Pretreatment of membranes with relatively low concentrations of N-ethylmaleimide inhibited [3H]OXO binding by 85% with no change in the Kd of [3H]OXO, and with no effect on [3H]quinuclidinyl benzilate binding.(ABSTRACT TRUNCATED AT 250 WORDS)