There is limited work exploring competency-based medical education (CBME) in undergraduate medical education. We aimed to assess medical students' and faculty's perception of CBME in the undergraduate medicine setting after its implementation at our institution through a Content, Input, Process, Product (CIPP) program evaluation model. We explored the rationale for the transition to a CBME curriculum (Content), the changes to the curriculum and the teams involved in the transition (Input), medical students' and faculty's perception of the current CBME curriculum (Process), and benefits and challenges of implementing undergraduate CBME (Product). A cross-sectional online survey was delivered over 8-weeks in October 2021 to medical students and faculty as part of the Process and Product evaluation. Medical students displayed greater optimism towards CBME, compared to faculty, in terms of its role in medical education (p<0.05). Faculty were less certain about how CBME was currently implemented (p<0.05), as well as how feedback to students should be delivered (p<0.05). Students and faculty agreed on perceived benefits to CBME implementation. Faculty time commitment to teaching and logistical concerns were reported as perceived challenges. Education leaders must prioritize faculty engagement and continued professional development of faculty to facilitate the transition. This program evaluation identified strategies to aid the transition to CBME in the undergraduate setting.