A previous study has shown that diverting attention from binocular rivalry to a visual distractor task results in a slowing of rivalry alternation rate between simple orthogonal orientations. Here, we investigate whether the slowing of visual perceptual alternations will occur when attention is diverted to an auditory distractor task, and we extend the investigation by testing this for two kinds of binocular rivalry stimuli and for the Necker cube. Our results show that doing the auditory attention task does indeed slow visual perceptual alternations, that the slowing effect is a graded function of attentional load, and that the attentional slowing effect is less pronounced for grating rivalry than for house/face rivalry and for the Necker cube. These results are explained in terms of supramodal attentional resources modulating a high-level interpretative process in perceptual ambiguity, together with a role for feedback to early visual processes in the case of binocular rivalry.