The purpose of this observational study was to compare the performance of a novel on-farm culture (OFC) test with the reference method (RM) in identifying pathogens, and in particular Staphylococcus aureus, associated with subclinical mastitis (SCM) in dairy cattle. The OFC test (Mastatest HiSCC; Mastaplex Limited) for SCM uses a cartridge with 2 × 12 wells allowing 1 sample to be analyzed in duplicate (24 wells) or 2 samples analyzed simultaneously, each in 12 wells. Results of the milk analyses are reported hierarchically (Staph. aureus → coagulase-negative staphylococci (CNS) → other gram positive or coliform/gram negative → no bacteria present) and emailed within 24 h. Milk samples (617 quarter level from 158 cows and 70 cow level) were collected from 288 cows [individual cow somatic cell count (ICSCC) ≥150,000 cells/mL] on 9 purposefully selected farms known to have a high prevalence of clinical and subclinical Staph. aureus mastitis in Southland New Zealand. Quarter samples were analyzed individually (617 samples) and after animal-level pooling, providing 228 (158 + 70) cow-level samples. Samples were analyzed by the OFC test (in duplicate) and the RM (culture agar medium and latex test based on the recommendation by the National Mastitis Council) and identifications confirmed with MALDI-TOF mass spectrometry. Sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) for detection of Staph. aureus were all ∼90% with tight 95% confidence limits, and Cohen's kappa (κ) for agreement between the OFC test and RM was 0.81. Kappa for agreement between the OFC test duplicates was 0.93. About 35% of cows had only one quarter infected with Staph. aureus and all these animals could still be identified when pooled cow-level milk was analyzed. Although the high prevalence of Staph. aureus in the herds used in this study does not affect the Se and Sp values, it does elevate the PPV value (and decrease the NPV) and therefore use of PPV to extrapolate to a population with lower prevalence is not appropriate. For CNS, Sp, PPV, and NPV were all >0.8, κ was ≥0.6, and Se was >0.7. Kappa for agreement between the OFC test duplicates was 0.83. A result of "no bacteria detected" was reported in 13% of the cows with 93% agreement between OFC test and RM. We conclude that the OFC test provides a reliable method for detecting Staph. aureus in pooled cow-level milk even if only one quarter is infected; in the absence of Staph. aureus in the milk, it reliably identified CNS in pooled cow-level milk; it reliably identified cows with <10 cfu/10 µL of their milk. Compared with the RM, the method was rapid with results returned in 24 h of loading the cartridge.