The prefrontal cortex appears to contribute to the mnemonic retrieval of the context within which stimuli are experienced, but only under certain conditions that remain to be clarified. Patients with lesions to the frontal cortex, the temporal lobe and neurologically intact individuals were tested for context memory retrieval when verbal stimuli (words) had been experienced across multiple (unstable context condition) or unique (stable context condition) contexts; basic recognition memory of these words-in-contexts was also tested. Patients with lesions to the right ventrolateral prefrontal cortex (VLPFC) were impaired on context retrieval only when the words had been seen in multiple contexts, demonstrating that this prefrontal region is critical for active retrieval processing necessary to disambiguate memory items embedded across multiple contexts. Patients with lesions to the left dorsomedial prefrontal region were impaired on both context retrieval conditions, regardless of the stability of the stimulus-to-context associations. Conversely, prefrontal lesions sparing the ventrolateral and dorsomedial regions did not impair context retrieval. Only patients with temporal lobe excisions were impaired on basic recognition memory. The results demonstrate a basic contribution of the left dorsomedial frontal region to mnemonic context retrieval, with the VLPFC engaged, selectively, when contextual relations are unstable and require disambiguation.