OBJECTIVE To determine whether anxiety and depression are associated with cognition in multiple sclerosis (MS), and whether these associations are similar in other immune-mediated inflammatory diseases (IMID; including inflammatory bowel disease [IBD] and rheumatoid arthritis [RA]) and in anxious/depressed individuals (ANX/DEP) without an IMID. METHODS Participants (MS: n = 255; IBD: n = 247; RA: n = 154; ANX/DEP: n = 308) completed a structured psychiatric interview, the Hospital Anxiety and Depression Scale, and cognitive testing, including the Symbol Digit Modalities Test, the California Verbal Learning Test, and Letter Number Sequencing test. Test scores were converted to age-, sex-, and education-adjusted z scores. We evaluated associations of anxiety and depression with the cognitive z scores using multivariate linear models, adjusting for disease cohort. RESULTS All cohorts exhibited higher rates of impairment (i.e., z less than or equal to -1.5) in the domains of processing speed, verbal learning, and delayed recall memory relative to general population norms. Higher levels of anxiety symptoms were associated with slower processing speed, lower verbal learning, and lower working memory performance (all p < 0.001); higher levels of depression symptoms were associated with slower processing speed. These associations did not differ across cohorts. CONCLUSIONS Anxiety and depression are associated with lower cognitive function in MS, with a similar pattern observed in persons with other IMID, including IBD and RA, and persons without an IMID. Managing symptoms of anxiety and of depression in MS, as well as other IMIDs, is important to mitigate their effect on cognition.