To evaluate whether cognitive performance is affected in newly diagnosed temporal lobe epilepsy (TLE) and to determine the most vulnerable cognitive domains. In this baseline longitudinal study, differences in memory and non-memory cognitive functions were assessed using comprehensive neuropsychological test batteries in 21 adult patients with newly diagnosed non-lesional TLE and individually matched controls. In addition, the analyses included ratings of self-perceived emotional status. The patients performed more poorly than the control group regarding delayed visual memory (p = 0.013) and executive function tasks related to switching (Trail Making Test and verbal fluency shifting; p = 0.025 and p = 0.03, respectively). We found no differences in verbal learning and memory, attention/working memory/processing speed, and other executive functions. Our results show that patients with TLE often have specific cognitive deficits at time of diagnosis, even in the absence of structural brain abnormalities. This supports the hypothesis that memory dysfunction is linked to an underlying pathology rather than to the effect of recurrent seizures, long-term use of anti-seizure medication, or other epilepsy-related factors. As certain executive functions are affected at an early stage, the pathology may involve brain regions beyond the temporal lobe and may comprise larger brain networks. These results indicate the need for greater awareness of cognition at the time of diagnosis of TLE and before initiation of treatment, and integration of neuropsychological assessment into early routine clinical care.