Detection of bulk tank milk (BTM) antibodies using ELISA (BTM-ELISA) may constitute an inexpensive test for surveillance of Mycobacterium avium subsp. paratuberculosis (MAP) infection in dairy cattle herds provided that the test is accurate and consistent. The objectives of this study were to determine: (a) the correlation between repeated BTM reactions; and (b) the association between the BTM antibody ELISA-level and the within-herd prevalence of antibody-positive cows. Eight BTM samples per herd and approximately four milk samples per lactating cow per herd were collected from each of 108 Danish Holstein herds over a period of one year. All samples were tested using a commercial indirect ELISA for detection of MAP specific antibodies. The individual cow's results were dichotomised and used to estimate the within-herd antibody prevalence at each test-date. These prevalences were then combined with the ELISA reading on the BTM test-date closest to the cow-level test-date. A mixed-effect analysis of covariance with autoregressive type 1 correlation structure was carried out using the log-transformed BTM-ELISA results as outcome. This model was used to assess the correlation between repeated tests with and without correction for within-herd antibody prevalence. The repeated BTM-recordings were highly correlated with a correlation of 0.80 between samples collected 1.5 months apart. The within-herd antibody prevalence significantly influenced this estimate (p<0.0001), which dropped to 0.60 when corrected for the within-herd antibody prevalence. Although the test-results were relatively consistent and correlated with the within-herd prevalence, the magnitude of the test-values makes it difficult to use the BTM-ELISA for surveillance of MAP infections in practice.