Rett syndrome is a neurodevelopmental disorder with potential for improvement through novel targeted therapeutics. Reliable outcome measures are critical to the development of treatments. We examined the merits and flaws of the Rett Syndrome Behavioural Questionnaire, an outcome measure for clinical trials. The Rett Syndrome Behavioural Questionnaire was administered alongside other clinical scales in three cohorts, an online survey, a clinic-based study, and the screening period for a clinical trial. Data were collected from individuals with Rett syndrome and related disorders at three time points, separated by a minimum of one week and a maximum of two months. We hypothesized that for clinical trial use, little change should occur among visits. Distribution statistics, internal consistency, intraclass correlation coefficient, percent agreement, and Cohen's kappa were examined. Among 149 with classic Rett syndrome, the Rett Syndrome Behavioural Questionnaire was completed 377 times. Median total score was 33, ranging from 3 to 73. Of the 51 items tested in the original Rett Syndrome Behavioural Questionnaire study, 24 exhibited either floor or ceiling effects. Friedman's analysis of variance revealed significant difference among visits (P = 0.024), and graphical analysis using Bland-Altman plots demonstrated systematic positive bias with a 95% confidence interval including up to 12.9 points higher to 15.7 points lower at retest. Median agreement measured by kappa was 0.53 for retest at visit 2 and 0.49 for retest at visit 3. The Rett Syndrome Behavioural Questionnaire did not achieve acceptable standards as an outcome assessment for clinical trials in Rett syndrome.