Competency-based education (CBE) programs usually evaluate student learning outcomes at a course level. However, a more comprehensive evaluation of student achievement of competencies requires evaluation at a programmatic level across all courses. There is currently insufficient literature on accomplishing this type of evaluation. In this article, we present an evaluation strategy adopted by the competency-based master's degree program at the Center for Health Professions Education at the Uniformed Services University of Health Sciences to assess student achievement of competencies. We hypothesized that (1) learners would grow in the competencies through their time in the program and (2) learners would exhibit a behavioristic change as a result of their participation in the program. The degree program at the Center for Health Professions Education conducts an annual student self-assessment of competencies using a competency survey. The competency survey data from graduated master's students were collected, providing data from three time points: initial (pre-program survey), middle, and final (end-of-program survey). Open-ended responses from these three surveys were also analyzed. A general linear model for repeated measures was conducted. Significant effects were followed by post hoc tests across time. We also conducted post hoc analysis across domains to better understand the comparative levels of the domains at each time point. The responses to the open-ended prompt were thematically analyzed. Analysis of the quantitative data revealed that (1) learners reported significant growth across time, (2) learners had different perceptions of their competencies in each of the domains, and (3) not all domains experienced similar changes over time. Analysis of the free responses highlighted the impact of coursework on competency attainment and the behavioristic change in learners. This study presents a strategic evaluation tool for course-based CBE programs that follow a traditional credit hour model. Programmatic evaluation of CBE programs should enable the inclusion of the learner's voice and provide evaluation data that go beyond individual course evaluations.