OBJECTIVE The Gaussian error function was first used and verified in normal tissue complication probability (NTCP) calculation to reduce the dose-volume histogram (DVH) database by replacing the dose-volume bin set with the error function parameters for the differential DVH (dDVH). METHODS Seven-beam intensity modulated radiation therapy (IMRT) treatment planning was performed in three patients with small (40 cm3), medium (53 cm3), and large (87 cm3) prostate volume, selected from a group of 20 patients. Rectal dDVH varying with the interfraction prostate motion along the anterior-posterior direction was determined by the treatment planning system (TPS) and modeled by the Gaussian error function model for the three patients. Rectal NTCP was then calculated based on the routine dose-volume bin set of the rectum by the TPS and the error function model. The variations in the rectal NTCP with the prostate motion and volume were studied. RESULTS For the ranges of prostate motion of 8-2, 4-8, and 4-3 mm along the anterior-posterior direction for the small, medium, and large prostate patient, the rectal NTCP was determined varying in the ranges of 4.6%-4.8%, 4.5%-4.7%, and 4.6%-4.7%, respectively. The deviation of the rectal NTCP calculated by the TPS and the Gaussian error function model was within +/- 0.1%. CONCLUSIONS The Gaussian error function was successfully applied in the NTCP calculation by replacing the dose-volume bin set with the model parameters. This provides an option in the NTCP calculation using a reduced size of dose-volume database. Moreover, the rectal NTCP was found varying in about +/- 0.2% with the interfraction prostate motion along the anterior-posterior direction in the radiation treatment. The dependence of the variation in the rectal NTCP with the interfraction prostate motion on the prostate volume was found to be more significant in the patient with larger prostate.