OBJECTIVE To develop and validate a clinical risk prediction score for noninvasive ventilation (NIV) failure defined as intubation after a trial of NIV in non-surgical patients. METHODS Retrospective cohort study of a multihospital electronic health record database. METHODS Non-surgical adult patients receiving NIV as the first method of ventilation within two days of hospitalization. METHODS Primary outcome was intubation after a trial of NIV. We used a non-random split of the cohort based on year of admission for model development and validation. We included subjects admitted in years 2010-2014 to develop a risk prediction model and built a parsimonious risk scoring model using multivariable logistic regression. We validated the model in the cohort of subjects hospitalized in 2015 and 2016. RESULTS Of all the 47,749 patients started on NIV, 11.7% were intubated. Compared with NIV success, those who were intubated had worse mortality (25.2% vs. 8.9%). Strongest independent predictors for intubation were organ failure, principal diagnosis group (substance abuse/psychosis, neurological conditions, pneumonia, and sepsis), use of invasive ventilation in the prior year, low body mass index, and tachypnea. The c-statistic was 0.81, 0.80 and 0.81 respectively, in the derivation, validation and full cohorts. We constructed three risk categories of the scoring system built on the full cohort; the median and interquartile range of risk of intubation was: 2.3% [1.9%-2.8%] for low risk group; 9.3% [6.3%-13.5%] for intermediate risk category; and 35.7% [31.0%-45.8%] for high risk category. CONCLUSIONS In patients started on NIV, we found that in addition to factors known to be associated with intubation, neurological, substance abuse, or psychiatric diagnoses were highly predictive for intubation. The prognostic score that we have developed may provide quantitative guidance for decision-making in patients who are started on NIV.