Many parasites exhibit antigenic variation within their hosts. We use mathematical models to investigate the dynamical interaction between an antigenically varying parasite and the host's immune system. The models incorporate antigenic variation in the parasite population and the generation of immune responses directed against (i) antigens specific to individual parasite variants and (ii) antigens common to all the parasite variants. Analysis of the models allows us to evaluate the relative importance of variant-specific and cross-reactive immune responses in controlling the parasite. Early in the course of infection within the host, when parasite diversity is below a defined threshold value (the value is determined by the biological properties of the parasite and of the host's immune response), the variant-specific immune responses are predominant. Later, when the parasite diversity is high, the cross-reactive immune response is largely responsible for controlling the parasitemia. It is argued that increasing antigenic diversity leads to a switch from variant-specific to cross-reactive immune responses. These simple models mimic various features of observed infections recorded in the experimental literature, including an initial peak in parasitemia, a long and variable duration of infection with fluctuating parasitemia that ends with either the clearance of the parasite or persistent infection.