Yield losses in sunflower crops caused by Plasmopara halstedii can be up to 100%, depending on the cultivar susceptibility, environmental conditions, and virulence of the pathogen population. The aim of this study was to investigate the genetic and phenotypic structure of a sunflower downy mildew agent at the field scale. The genetic diversity of 250 P. halstedii isolates collected from one field in southern France was assessed using single-nucleotide polymorphisms (SNPs) and single sequence repeats (SSR). A total of 109 multilocus genotypes (MLG) were identified among the 250 isolates collected in the field. Four genotypes were repeated more than 20 times and spatially spread over the field. Estimates of genetic relationships among P. halstedii isolates using principal component analysis and a Bayesian clustering approach demonstrated that the isolates are grouped into two main genetic clusters. A high level of genetic differentiation among clusters was detected (FST = 0.35), indicating overall limited exchange between them, but our results also suggest that recombination between individuals of these groups is not rare. Genetic clusters were highly related to pathotypes, as previously described for this pathogen species. Eight different races were identified (100, 300, 304, 307, 703, 704, 707, and 714), with race 304 being predominant and present at most of the sites. The co-existence of multiple races at the field level is a new finding that could have important implications for the management of sunflower downy mildew. These data provide the first population-wide picture of the genetic structure of P. halstedii at a fine spatial scale.