Retroactive interference occurs when new information disrupts the retention of an existing representation, but its effects on visual short-term memory remain poorly understood. The present study examined three factors predicted to influence domain-specific retroactive interference, including the type of distractor, its temporal position, and the length of the retention interval. Participants compared target and test objects over a brief interval that either was unfilled or contained a similar or dissimilar distractor occurring 200 ms or 1.5 s after the target offset. Retention was influenced by the temporal position of the distractor and its relationship with the to-be-remembered target. Specifically, retroactive interference was only observed following the presentation of a dissimilar distractor that occurred 1.5 s after the target. These results suggest that novel distractors may be particularly interfering.