In Ayurveda, pulse diagnosis and body constitution diagnosis have a long historical use; still, there is lack of quantitative measure of the reliability of these diagnostic methods. Reliability means consistency of information. Consistent diagnosis leads to consistent treatment and is important for clinical practice, education, and research. The objective of this study is to study the methodology to evaluate the test-retest reliability (repeatability) of pulse diagnosis and body constitution diagnosis. A double-blinded, controlled, clinical trial was conducted in Copenhagen. The same doctor, an expert in Ayurvedic pulse diagnosis, examined the pulse and body constitution of 17 healthy participants twice, in random order without seeing them. A metric on pulse and body constitution variables was developed. Cohen's weighted kappa statistic was used as a measure of intra-rater reliability. Permutation tests were used to test the hypothesis of homogeneous diagnosis (ie, the doctor's diagnosis does not depend on the subject). The hypothesis of homogeneous classification was rejected on the 5% significance level (P values of .02 and .001, respectively, for pulse and body constitution diagnosis). According to the Landis and Koch scale, values of the weighted kappa for pulse diagnosis (P = .42) and body constitution diagnosis (P = .65) correspond to "moderate" and "substantial" agreement, respectively. There was a reasonable level of consistency between 2 pulse and body constitution diagnoses. Further studies are required to quantify inter-subject and intra-subject agreement for greater understanding of reliability of pulse and body constitution diagnosis.