Incisional hernia (IH) is the most frequent complication after abdominal surgery. Long-term follow-up is crucial. Patient-reported outcome measurements (PROMs) are able to monitor patients' disease progression after treatment. Until now, there are no PROMs that assess patients after abdominal surgery or that detects patients with IH. We aimed to develop a reliable questionnaire to assist in diagnosing IH, called the "PROMIS questionnaire": Patient-Reported Outcome Measurements in the Diagnosis of Incisional Hernias. In this pilot study, the reliability of this questionnaire is being determined. Patients diagnosed with IH between 2013 and 2014 were included. A questionnaire with seven questions was developed. Patients were asked whether they thought they had IH, whether they felt any pain at the site of the scar, and whether they saw or felt a lump or a bulge. Furthermore, smoking history and patients' weight and height were taken into account. Patients were approached three times by telephone, with an interval of 1 week. Test-retest reliability, internal consistency, and sensitivity were measured. Forty-three patients were included. Test-retest reliability was 1.0, and internal consistency was 0.56. The question regarding patients' pain was least consistent with other questions. The overall sensitivity of the questionnaire was 95%. The PROMIS questionnaire is a highly reliable questionnaire, but the internal consistency is modest. The clinical relevancy of pain in IH patients is essential. Therefore, this question will be kept in the current PROMIS questionnaire. It needs further validation in a prospective cohort study, to use it as a diagnostic tool in the future to detect IH.