The literature contains conflicting results concerning whether an irrelevant featural singleton (an item unique with respect to a feature such as color or brightness) can control attention in a stimulus-driven manner. The present study explores whether target-nontarget similarity influences stimulus-driven shifts of attention to a distractor. An experiment evaluated whether manipulating target-nontarget similarity by varying orientation would modulate distraction by an irrelevant feature (a bright singleton). We found that increasing target-nontarget similarity resulted in a decreased impact of a uniquely bright object on visual search. This method of manipulating the target-nontarget similarity independent of the salience of a distracting feature suggests that the extent to which visual attention is stimulus-driven depends on the target-nontarget similarity.