Lessons learned from real-world implementation of entrustable professional activities in an internal medicine residency programme

ABSTRACT

The competency-based medical education framework has driven the adoption of entrustable professional activities (EPAs) for workplace-based assessment, yet real-world implementation experience from Southeast Asian settings remains scarce. In this commentary, the authors describe the transition to an EPA-based assessment model in an internal medicine residency programme in Singapore, drawing on perspectives from the programme director, core faculty members, programme coordinator, and graduates who participated in a roundtable discussion. Six key lessons emerged. First, explicit and repeated dialogue about trust is needed to secure buy-in from learners and assessors. Second, EPA tasks and standards must be clearly linked to core professional competencies while balancing assessment burden through simplified, distributed workplace-based assessments. Third, informal feedback from supervisors, though often undocumented, carries valuable performance information that should be systematically captured. Fourth, overall entrustment decisions require holistic synthesis of quantitative and qualitative data rather than simple score aggregation. Fifth, mapping entrustment gaps back to underlying competency domains enables targeted remediation. Sixth, there continues to be a need to address the key limitations of current EPA models, which include inter-rater variability and misalignment with fixed-length residency training structures. These practical insights offer guidance for programmes navigating similar transitions and highlight the need for further qualitative research to refine EPA implementation strategies in comparable training contexts.

In the early 2000s, the competency-based medical education (CBME) framework became the gold standard for clinical training and assessment of physicians in postgraduate subspecialty programmes. These physicians were regularly assessed on Accreditation Council for Graduate Medical Education (ACGME) core competencies (patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice) to determine their eligibility for training progression. Subsequently, workplace-based assessments (WBAs) of entrustable professional activities (EPAs)^1,2 became popularised, where desirable core competencies are mapped to practical work descriptors, such that entrustment decisions can be made by preceptors based on the assessed level of supervision required by the trainees to carry out professional tasks.¹ While the application of EPAs has been described extensively in Western healthcare systems from which it was first developed,^3,4 there is a relative paucity of literature exploring its real-world implementation in Asian contexts,⁵ where there could be unique sociocultural and practice-specific nuances and challenges.  

In this article, the authors, comprising the programme director (PD), 2 core faculty members (CFMs), a residency programme coordinator, and 2 graduates of an internal medicine (IM) residency programme in Singapore, came together for a roundtable discussion of their experiences on the ground holding different roles within the EPA training system, and distilled key lessons learned and challenges in real-world EPA implementation. 

In 2010, Singapore first launched its ACGME-accredited residency programmes for postgraduate specialist training.⁶ By 2023, residency programmes in Singapore started to transition to a revised training model designed to translate broad competencies into EPAs.^2,7 Likewise, in the National University Health System (NUHS), the residency programmes gradually transitioned to an EPA-based assessment model between 2023 and 2025.

The NUHS IM residency programme is a 3-year postgraduate specialty training programme that hosts, on average, 100–125 junior residents across training sites within the cluster (National University Hospital, Ng Teng Fong General Hospital, and Alexandra Hospital). The programme is supported by 29 CFMs who provide longitudinal supervision, mentorship, and performance review to their assigned resident mentees. WBAs are regularly conducted, including the mini-clinical evaluation exercise, chart-stimulated recall, direct observation of procedural skills, multisource feedback, monthly end-of-rotation evaluations, and ambulatory care assessments. Unlike older WBAs that involved evaluation of generic competency domains, the current EPA framework rates trainees’ performance in relation to practical units of work under 4 core IM EPAs (inpatient care, ambulatory care, acutely ill patient, and bedside procedures). The level of entrustment for each assessment item is accorded based on observed clinical performance and entrustment-based discussion, with higher entrustment levels corresponding to a reduced need for supervision.

Every 6 months, residents will consolidate their portfolio comprising an aggregate of WBAs and extra-clinical achievements before meeting their assigned CFM for a joint review and discussion. The number of WBAs required in relation to the respective EPAs for each resident is outlined in Table 1. Where additional context is required, CFMs may seek qualitative feedback from supervising physicians. Thereafter, the CFM proposes an overall entrustment level for their assigned resident at a central clinical competency committee (CCC) meeting. CCC faculty members will deliberate on the final entrustment levels for the 4 core IM EPAs after rounds of robust discussion grounded in the data and feedback provided within the WBAs, in alignment with the best practices advocated for programmatic assessments in CBME.⁸ Advancement to subsequent phases of training is contingent on the attainment of pre-defined minimum entrustment thresholds appropriate for the level of training (Fig. 1). 

Table 1. Tabulation of the required WBAs mapped to EPAs in the internal medicine residency programme at NUHS.

Fig. 1. Visual guide on EPA-based competency appraisal model in the internal medicine residency programme at NUHS.

Lessons learned from real-world EPA implementation

Communicate a shared understanding of trust explicitly with stakeholders (learners and assessors)  

The introduction of EPAs necessitates deliberate and transparent conversations about trust,¹ which is the guiding principle that links observed performance to graded autonomy, in order to secure buy-in from stakeholders (i.e. learners and assessors) to this model of competency appraisal.

At a broader level, entrustment in medical care relates to the fiduciary principle of the patient–physician relationship.⁷ In tertiary hospital settings, specialist consultants bear the ultimate responsibility for patient care, but are supported by a team of junior physicians and residents-in-training to whom they entrust various clinical duties in executing decisions and patient management. 

Additionally, an entrustment-based model more appropriately appraises the longitudinal progress of attaining entrustment milestones in professional tasks than older frameworks, which focus on assessing absolute competency that could be unrealistic, especially for advanced skill sets.⁷ Sustaining this shared understanding requires repeated and structured engagement with stakeholders through faculty development workshops, dialogue, and periodic review of on-the-ground EPA implementation.

Within the Singapore context, training was conducted at various specialty departments across all sites and reached approximately 250–300 physician faculty members prior to the transition to EPAs in July 2023. These sessions presented opportunities for regular conversation between the residency programme and clinical departments across training sites to sustain a shared understanding of what contributes to grounded trust. Both assessors and residents were reminded of specific tasks tied to EPAs while the degree of supervision related to varying levels of entrustment were reiterated. Given the limited ability to achieve a clear understanding of EPA and the related entrustment concepts in a single sitting, constant engagement was delivered repeatedly in bite-sized content (e.g. short presentations, reminders via messaging platforms, and emails) 2–3 times a year. These methods created and sustained a shared understanding of grounded trust.

Relating EPA tasks and standards to core professional competencies: Practical execution of WBAs 

In general, EPAs must form part of standard professional work in a given context and represent a recognisable clinical work output that can be assessed in an observable and measurable manner. This work is performed via a demonstration of knowledge, skill sets, and attitudes that can be acquired through training and is executed within a stipulated timeframe under varying degrees of supervision.² Importantly, they must relate to 1 or more underlying core professional competencies.² Within an IM residency programme, EPAs such as “taking a history from a newly admitted patient from the emergency department” or “managing an acutely deteriorating patient” may relate to multiple ACGME core competencies, such as patient care, medical knowledge, and interpersonal/communication skills.

Additionally, the need for comprehensive EPA assessments must be carefully balanced against assessment fatigue that can lead to inaccurate appraisals. Therefore, EPA forms with simplified descriptors (and illustrated examples) were employed, with each evaluation encounter assessing only a limited aspect of clinical work (e.g. 1 assessment form allowing a maximum of 3 EPAs to be assessed). Assessment burden can also be split among a larger pool of assessors to allow for diversity of opinions and reduce the risk of bias. For example, inpatient and ambulatory care EPAs on clinical skills are assessed by covering consultants, while competency domains such as professionalism and communication are evaluated by nurses and allied health colleagues. Importantly, conduct of assessments can be flexible based on the clinical supervisors’ discretion—for instance, an EPA form might be filled out retrospectively if the preceptor has opportunistically observed a resident performing certain clinical tasks during regular ward work.⁹ To balance the risk of heterogeneity of opinions and EPA scale utilisation by a larger pool of assessors, the authors limit the assessor group for specific assessments, such as the chart-stimulated recall, to appointed CFMs to improve the reliability of EPA grading.¹⁰

Harnessing valuable performance datapoints outside of structured assessments

In practice, some of the most consequential information about trainee performance emerges outside formal assessment tools. Unsolicited, spontaneous feedback from clinical supervisors is frequently received informally by the PD, CFMs, or programme coordinators to highlight concerns of suboptimal clinical practice or professional conduct. The tendency of faculty supervisors refraining from formally documenting negative observations in writing is driven by their fear of limiting trainees’ progression and desire to avoid potentially awkward interactions with junior staff whom they may work with again in future. There is also a risk that informal feedback becomes perceived as gossip, which can then damage the institutional feedback culture. However, from the authors’ experience, informal feedback often contains rich information, which can become valuable in driving performance improvement if leveraged in a constructive manner.¹¹

To harness this important source of data while maintaining fairness and reliability, programme directors and CCC chairpersons must engage supervisors in understanding the principles of programmatic assessment.¹² Faculty members need clarity on how individual data points over a longitudinal period contribute to overall entrustment decisions and how timely, direct feedback can benefit trainee development. When supervisors appreciate that entrustment determinations rely on aggregated evidence rather than isolated comments, the barrier to feedback may be reduced as they feel more assured that raising concerns is constructive rather than punitive.

Trainees will also benefit from opportunities to learn from direct and real-time feedback and have the chance to provide contextual information to account for their observed performance; factors such as fatigue, multitasking, and distractions, and varying institutional practices and norms can have significant impact on trainees’ abilities to perform in the real world.¹³ Such valuable information may be lost if trainees do not have the chance to provide clarification. Data obtained from informal sources must be clarified and triangulated via the described measures and clearly documented within portfolios before they can be used in the determination of entrustment levels for the respective EPAs. In NUHS, CFM supervisors are responsible for such data verification and triangulation work before portfolios are presented at CCC to help ensure that the entrustment decision-making process is grounded in robust datapoints.

Pooling data for overall entrustment appraisal: More than the sum of its parts

There is an inherent risk that residency programmes may default to summing or averaging individual EPA scores to derive overall entrustment levels. This approach is unsound given that individual entrustment scales are ordinal and non-continuous in nature,¹⁴ such that the overall entrustment decisions cannot be a summation of individual assessments. Additionally, qualitative feedback provided in each entrustment assessment encounter must be considered, as it offers rich contextual information on how the entrustment decision was made and provides insights into possible contributing factors to trainees’ demonstrated performance.

Meaningful entrustment decisions therefore require deliberate and thoughtful synthesis by CFMs. Prior to CCC meetings, faculty supervisors must review the full portfolio, integrate numerical ratings with narrative comments, and consider the breadth of clinical exposure represented. Examination of case mix and contextual factors is particularly important, as entrustment in 1 domain may not be generalisable to all clinical situations. 

Eventually, each CFM should present an interpretive narrative of the resident’s developmental trajectory, proposing entrustment levels grounded in the holistic appraisal of the evidence (based on the previously described methods). Central CCC deliberations then serve as a forum for further calibration and peer scrutiny. Technology can support this process through information systems that tabulate assessments, visualise trends, and aggregate qualitative comments to enhance clarity and efficiency.¹⁵

Analysing competency deficiencies from entrustment decisions to guide feedback and remediation

Entrustment levels describe degrees of supervision required in performing professional tasks, but do not specify the underlying causes of underperformance. When residents fail to achieve expected EPA levels, it is essential to analyse the contributory competency gaps. Low entrustment in inpatient care, for instance, may stem from deficiencies in foundational medical knowledge, inappropriate clinical judgement within the practice context, or poor communication skills. Mapping entrustment outcomes back to ACGME core competency domains is therefore necessary for targeted feedback and structured remediation.

Going a step further, specific frameworks help provide granular analysis of performance by looking for qualities or the lack thereof, which influence entrustment decisions. CFMs look for behaviours related to attributes such as agency, reliability, integrity, and humility, which may not be completely captured within the traditional competency models.¹⁶ This helps to address the core question of why the trainee in question cannot be entrusted to perform specific tasks (defined within EPAs) at a stated level of supervision (e.g. direct or remote). It will also help the programme articulate the desired behaviours and attitudes that must be demonstrated to help trainees become trustworthy.

Furthermore, supervisors should examine whether identified competency gaps are pervasive across multiple EPAs or confined to specific contexts. A resident who performs competently in scheduled ambulatory clinics but struggles in acute emergency settings may require focused support in managing time pressure, prioritisation, or crisis communication. Conversely, cross-cutting concerns across several EPAs may indicate a more foundational deficiency. Specific remediation strategies can then be planned to close these gaps in the appropriate work environment context defined by the corresponding EPAs.

Limitations of EPA models in present residency constructs

First, EPAs may be subject to bias (e.g. pre-conceived notions about certain trainees) and inter-rater variability (e.g. assessor idiosyncrasies, “hawk”/“dove” assessors, differing levels of expertise in task appraisals, and variable frame of reference).¹⁷ Moreover, after migrating the assessment construct from traditional competency scales to EPA-based scoring, the authors recognised that faculty members may have differing interpretations of how grading should be done. For example, while 1 might appraise a high-performing junior resident at an entrustment level of a registrar (indirect/remote supervision), others may accord this individual with an entrustment grade that simply matches the current level of training, thereby under-appraising the individual’s entrustment level. Strategies to mitigate these drawbacks include faculty development programmes, objective deliberation by a central assessment committee not directly involved in supervisory roles, and regular audits of the grading patterns of individual assessors. Specifically, assessors must be reminded to go beyond vague adjectival qualifiers (e.g. diligent, astute, careless, incompetent, etc.) and instead provide thick description of observed performance and behaviour to aid the CCC in grading the overall performance in relation to the pre-defined entrustment levels.

In addition, when Professor Olle ten Cate conceived the EPA model, he envisioned the concurrent need to evolve medical training from “fixed-length variable-outcome programmes” to “fixed-outcome variable-length programmes”.² Yet, in reality, residency programmes operate within a time-bound model where residents can only progress to the next phase of training after completing a stipulated duration of clinical postings. Therefore, while some residents who fail to meet entrustment cut-offs may require remediation postings and extension of residency training, it remains impossible within the present structure for high-achieving residents who attain higher levels of entrustment to “fast-track” their training. How the residency structure can be adjusted to arrive at a fixed-outcome variable-length model commensurate with the ethos of an EPA assessment framework warrants further discourse.

CONCLUSION

The authors shared their key takeaways from on-the-ground experiences with the implementation of an EPA model of professional assessment in an IM residency programme in Singapore. Moving forward, formal qualitative studies involving interviews and focus group discussions with key stakeholders of EPA implementation in Singapore residency programmes would be helpful to iron out the common challenges and implementation strategies that can optimise the EPA model of competency assessment.

REFERENCES

1. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ 2013;5:157-8.
2. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ 2005;39:1176-7.
3. Burke AE, Sklansky DJ, Haftel HM, et al. Competency-based medical education and the education continuum: Establishing a framework for lifelong learning. Curr Probl Pediatr Adolesc Health Care 2024;54:101642.
4. Hennus MP, Pinilla S, Touchie C, et al. Implementing entrustable professional activities: Practical lessons and legal considerations from an international interview study. Med Teach 2026;48:794-802.
5. Shorey S, Lau TC, Lau ST, et al. Entrustable professional activities in health care education: a scoping review. Med Educ 2019;53:766-77.
6. Huggan P, Samarasekara D, Archuleta S, et al. The successful, rapid transition to a new model of graduate medical education in Singapore. Acad Med 2012;87:1268-73.
7. Ng IKS, Mok SF, Teo D. Competency in medical training: current concepts, assessment modalities, and practical challenges. Postgrad Med J 2024;qgae023.
8. van der Vleuten CPM, Schuwirth LWT, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach 2012;34:205-14.
9. Lam R. A three-step approach to enhancing the educational value of workplace-based assessments. Med Teach 2025;47:902.
10. Reddy ST, Endo J, Gupta S, et al. A Case for Caution: Chart-Stimulated Recall. J Grad Med Educ 2015;7:531-5.
11. Chiel L, Brouwer E, Fishman MD, et al. “It’s hard not to think of feedback as gossip”: exploring resident perspectives on the relationship between gossip and feedback. Acad Med 2026;101:1005-11.
12. van der Vleuten CPM, Lindemann I, Schmidt L. Programmatic assessment: the process, rationale and evidence for modern evaluation approaches in medical education. Med J Aust 2018;209:386-8.
13. Lipnevich AA, Mattern K, Feddock C. Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189. Med Teach 2026;48:921-40.
14. Ten Cate O. When I say … entrustability. Med Educ 2020;54:103-4.
15. Desai SV, Khan S, Lomis K. AI-Enabled Precision-Education Systems – Transforming Lifelong Learning in Medicine. N Engl J Med 2026;394:838-41.
16. Ten Cate O, Chen HC. The ingredients of a rich entrustment decision. Med Teach 2020;42:1413-20.
17. Wilma Kleijer EF, Pool IA, Schuurmans MJ, et al. Subjectivity in Entrustment Decision-Making in the Clinical Workplace: Can We Retain Its Legitimacy But Avoid Bias? Med Sci Educ 2026;36:571-8.