Designing the Future of Medical Education: The EPA Framework as a Catalyst to Inform Family Planning and Reproductive Health Fellowship Training Program in Ethiopia Medical Education: An Exploratory Sequential Mixed Method Study

Introduction

Professional education has not been aligned with emerging health challenges, as it frequently yields graduates who lack the necessary qualifications. To effectively impact health outcomes, the professional education sub-system must devise novel educational and institutional strategies. This entails embracing competency-based approaches to enhance education and swiftly adapting them to the ever-evolving global landscape.¹

The implementation of competency-based medical education education (CBME) would better prepare medical graduates to effectively navigate complex scenarios.^2,3 In recent years, a notable transformation has taken place in medical education programs. This shift has been primarily influenced by the widespread adoption of competency-based education, which has revolutionized the way medical professionals are trained.^4–6

Many countries have made different frameworks for CBME.^7–10 These frameworks serve as invaluable guides that facilitate the global implementation of this approach. The Can-MEDS framework is a widely utilized competency framework that encompasses seven essential roles for doctors, regardless of their specialization. The seven roles include 1) medical expert, 2) communicator 3) collaborators, 4) leader, 5) advocator 6) scholar and 7) professional. This framework is employed on a global scale, including in Ethiopian medical education, to guide the development of curriculum.^11–16

While competency frameworks are valuable for designing competency-based medical education programs, reports have emerged of medical educators facing challenges in implementing these competencies in their everyday work. The focus of these competencies lies mainly on describing the qualities and attributes of physicians, placing less emphasis on specific duties and obligations.^17,18 The complexity of these competencies makes it difficult to create an effective training program. As a result, this makes them overly hypothetical to undergo training and be accurately evaluated.^17,18

Moreover, it is crucial to highlight that the evaluation of these competencies is carried out autonomously, disregarding the comprehensive range of roles expected from a proficient specialist.^11,19 Moreover, the absence of a reliable information source hinders the ability to accurately gauge progress and establish a solid basis for making summative entrustment decisions. Informal observation is used to make summative judgments about a trainees performance.^20,21.

In order to fully implement Competency-Based Education (CBE) and ensure that competencies are applicable to everyday clinical practice, it is crucial to translate these theoretical competencies into practical tasks and responsibilities that can be carried out by trainees without supervision (ie Entrustable Professional Activity (EPA)^{6,14,17,20,22–27} EPAs are units of professional practice, comprising responsibilities or tasks that can be delegated to a student for independent execution once they have acquired the necessary competence. EPAs serve as a link between competencies and clinical context, enabling the translation of competencies into meaningful, trainable, and assessable skills for clinical teachers.^{6,14,17,20,22–26}

EPAs have been proposed in a number of specialized and sub-specialized programs.^{5,6,16,27–31} and has gained popularity in medical education.^14,16 Due to disparities in professional practice between different contexts, the automatic transfer of a set of core EPAs is not feasible. Furthermore, the scarcity of EPA studies conducted in African and Asian countries highlights the need for research studies. Hence, the main objective of the study was to create an EPA framework that can effectively inform the Family Planning (FP) and Reproductive Health (RH) Fellowship Program within the unique context of Ethiopian health professionals training.

Methods and Materials

Study Design

In our study, we employed an exploratory sequential mixed-method design that included the collection of both qualitative and quantitative data. Initially, we collected data from a group of subject matter experts during the qualitative phase. Subsequently, we meticulously analyzed the results, leading us to develop new instrument, specifically a new list of EPAs, based on these findings. Finally, we administered the newly devised instrument to these experts, ensuring its applicability in their future work.

Study Setting and Target Population

This research was undertaken in all Obstetrics-Gynecology (Ob-Gyn) residency training institutions across the country. According to the October 2022 data from the Federal Ministry of Health Residency Program, there were a total of 13 institutions providing residency training in the field of obstetrics and gynecology. These institutions offer a 4-year residency training program in the Ob-Gyn specialty as well as a 2-year sub-specialist training program in Family Planning and Reproductive Health Fellowship. The target population consisted of obstetrics and gynecology (Ob-Gyn) specialists and sub-specialists from all 13 institutions that offer residency training in the field of obstetrics and gynecology across the country.

Data Collection Technique

Qualitative Data

We utilized a Nominal Group Technique (NGT) with few experts to identify potential EPAs and to establish the relevance of each candidate EPAs. The NGT is a structured approach that encourages active participation from all participants, ensuring that their opinions and ideas are heard and considered.^32,33 Delbecq and Van de Ven created it, consisting of four crucial phases: quiet generation, round robin, elucidation, and balloting.³³ Also, because just asking a few experts might not be good enough to make sure these EPAs are important,^21,34 the survey methods were conducted to further validate candidate EPAs and to ensure that these EPAs are actually used in real jobs and that the people who use them think they are helpful.

Procedure

NGT Phase 1: Assembly of Experts and Suggestion of Candidate EPAs

The assembly of this NGT was guided primarily by professional expertise. The panelists for this round were deliberately chosen because of their experience and active participation in residency training programs. The panel in this study included four Ob-Gyn specialists as well as three RH sub-specialists from four residency training institutions. At the beginning of the meeting, the principal investigator (PI) who was expert on EPAs was given a 30 minute lecture on the concept and key characteristics of an EPAs. After a thorough discussion on the key attributes of EPAs, participants were given up 40 minutes to silently generate and propose potential end-of-training EPAs.³²

The PI then asked one participant at a time to state a single potential EPA to the panelists in a “round robin” manner without any discussion. The PI meticulously documented these ideas on an Excel spreadsheet.³³ To ensure participant understanding, a serial discussion of potential EPAs (ie taking each potential EPA, one at a time (serially) and discussion were conducted.³³

NGT Phase 2: Preliminary Voting of Each Proposed EPAs

During this phase, the panelists were individually tasked to judge the importance of every candidate end of training EPA. The rating was done on a four-point scale, ranging from 1 (not important) to 4 (very important). This evaluation took into consideration both the criticality of the task and the frequency with which specialists encounter it in an authentic work setting. In the final step of this phase, individual judgments on the relevance of EPAs are combined into a group decision. In this particular step, the relevance rating was recoded as follows: a rating of 1 indicated an importance scale of 4 or 3, while a rating of 0 indicated an importance scale of 2 or 1 (Table S1 Supplementary File).

In order to assess the relevance and importance of each proposed EPA, we employed the content validity indexes (CVIs). These indexes were calculated based on the recommended guidelines outlined by Lynn³⁵ and Polit & Beck³⁶ (Table S2 Supplementary File). Candidate EPA with an item-level content validity index (I-CVI) and scale-level content validity index (S-CVI/Ave) of 0.83% or higher for six to eight experts is considered acceptable and applicable^16,35,37 (Table S3 Supplementary File). After calculating the I-CVI and S-CVI/Ave, we compiled a final list of EPAs. Subsequently, we presented this list along with the corresponding ratings to a panel of experts. This provided them with an opportunity to reassess their initial opinions and judgments. To ensure ample time for thorough evaluation, a deadline of two weeks was set for the completion of this round.

Quantitative Data Collection

Survey

The nationwide survey was carried out between July and August 2022, with the goal of further validating whether the candidate EPAs are supported by those who work with them and are truly part of work. We invited all obstetrics and gynecology specialists, including sub-specialists, from the 13 residency training institutions to actively participate in our nationwide survey. During this phase, participants were asked to provide their opinions on the candidate EPAs using a scale ranging from 1 to 5, which is commonly employed in various medical education contexts.^5,16,35 The scale allowed them to express their level of agreement, which ranged from strongly agreeing to strongly disagreeing. The questionnaire also included questions about the participants’ clinical and academic experiences, as well as their socio-demographic information. In order to determine the validity and acceptability of the scale instrument, an I-CVI of 0.80 or higher and an S-CVI of 0.90 or higher were considered the standard criterion for these data.^16,35,38

Data Analysis

Qualitative data analysis involved several steps, including compiling a list of tasks, removing duplicate tasks, reviewing candidate EPAs using criteria and an equal rubric tool, reviewing the literature to add potential EPAs, and developing a new list of EPAs for the second round quantitative phase (survey). Descriptive statistics, validity index analysis, and intra-class correlation coefficients were used for quantitative data analysis.

Ethical Considerations

The study underwent review and received approval from the Institutional Review Board (IRB) of St. Paul Hospital Millennium Medical College. The approval number is PM23/297. The IRB ethics committee confirmed that verbal informed consent was deemed acceptable. The information sheet was read and explained to each study participant by the data collectors. Subsequently, the participants were requested to give their verbal consent to take part in the research. It is worth noting that the participants willingly gave their oral consent to take part in this study, as it posed no significant risks or consequences.

Result

Characteristics of Expert Panel (NGT Phase 1 and 2)

All the invited panel members (n = 7) consented to participate and all completed the first and second phase of NGT (100% response rate). Three of the seven panels (43%) were Ob-Gyn specialist and sub-specialist in family planning and reproductive Health, and the panel consisted mostly of male participants, accounting for 57% of the total. On average, the panelists had been practicing for 10.3 years, with a range of 8 to 13 years. All panelists are actively engaged in educating residents and delivering clinical services in Department of Obstetrics & Gynecology (Table 1).

Table 1 A Characteristics of the Panelist in NGT Phase 1 and 2 (n = 7)

Phase I NGT: Suggestion of Candidate EPAs (Exploratory Qualitative Phase)

During the first qualitative phase of this study, seven experienced panelists were able to propose a total of 57 candidate EPAs that must be performed by beginner level RH sub-specialist fellow independently. On average, each expert suggested 8 EPAs, ranging from a minimum of five to a maximum of eleven (Table S4 Supplementary File).

After analyzing qualitative data in phase one, we summarized the proposed candidate EPAs and rephrased them. We also grouped and categorized similar professional units of work and modified the EPAs using criteria and tools. Additionally, we added some new tasks based on literature. As a result, a new list of potential EPAs was developed, and the number of remaining candidate EPAs is now 17 (Table 2).

Table 2 List of the Potential End of Family Planning and Reproductive Health Fellowship Training EPAs After Analysis of Qualitative Data at Phase 1 NGT

Phase II NGT: Content Validation of Candidate EPAs

Seven expert panelist participated in NGT phase one (n=7) completed the second phase (100% response rate). Every single one of the seventeen EPAs was judged as very important or important. They all achieved acceptable item-level content validity index (I-CVI > 0.83). Out of those, 14 EPAs had all the experts agreeing on them (S-CVI/UA = 1.00) (Table 3).

Table 3 The Relevance/Importance Ratings of EPAs by Seven Experts

Phase III: EPA Validation Using Survey

The study entailed gathering data from a sample of 95 experts, resulting in an impressive response rate of 76% of the total number of experts invited from the Ethiopian Society of Obstetrics and Gynecology (125). It is worth noting that the majority of respondents were male. Twelve (12.7%) of the study participants were sub-specialized, while the rest were ob-gyn specialists. Furthermore, more than half of the participants had five years or more professional experience (Table 4).

Table 4 Socio Demographic Characteristics of Study Participants in the Survey

The final validation survey included all the seventeen EPAs with CVI of 83% and higher in phase two. All EPAs (100.0%) met the minimum standard criterion for acceptability (I-CVI > 0.83) as judged by panel of experts. Furthermore, the entire scale-level content validity index (S-CVI/Ave) of the EPAs was found to be 0.91 (Table 5).

Table 5 The Relevance Ratings of EPAs by 95 Experts

Furthermore, expert values showed a high Intra-Class Correlation (ICC) among the 95 experts (ICC = 0.815 with a 95% CI [0.756,0.865] P value 0.0001) (Table 6).

Table 6 Inter-Rater Agreement for the Expert Panel (N = 95)

Discussion

The objective of this study was to create a credible and acceptable set of EPAs for RH sub-specialty training program. This study represents a pioneering effort to introduce EPAs as a transformative guide for curriculum development, teaching methodologies, and competency assessment within the Family Planning and Reproductive Health Fellowship training program in Ethiopian medical education.

During the course of this research, a comprehensive set of 17 approved and acknowledged end-of-training EPAs was discovered and formulated through a collaborative process involving experts. Determining EPAs as suitable units of professional practice typically involves a collaborative and iterative approach among professionals.²¹ The initial step in the EPA identification process entails assembling a panel of experts. These highly knowledgeable individuals are then entrusted with the responsibility of identifying the core professional tasks and responsibilities that trainees can proficiently execute without supervision. The EPA identification process in this study was consistent with the methods commonly used in similar studies conducted in various settings.^4,16,39,40

Through this iterative approach, a final agreement (high Intra-class Correlation Coefficient (ICC) was achieved on 17 core EPAs. A high ICC indicates a high degree of similarity (agreement) between values from the expert groups. This suggest that these EPAs hold significant importance in showing what professionals do in real life, seamlessly integrating into real-world scenarios, and effectively adapting to the local context.^16,36,41

EPAs with high I-CVI >0.83 indicate that these EPAs are essential professional practice to the discipline. The entire (S-CVI/Ave) of EPAs 0.91 indicates that the EPAs meet the standard for being acceptable. The EPA validity index of 1.0 obtained from the survey of 95 respondents, in particular, indicated the suitability for use among educators who will work with these EPAs in the future. In addition, the amount of EPAs attained in the final expert consensus of the study aligns with the number suggested for postgraduate programs (ie 15–30).^22,42,43

An EPA is expected to have a profound effect on the programme, trainers, trainees, and the community.⁴⁴ We anticipate that our EPA framework will have a direct impact on training and, consequently, lead to improved outcomes in patient and family care. The implementation of EPAs in the program signifies a significant transformation in the structure of training programs. This shift entails moving away from a time-dependent approach to one that focuses on outcomes, as well as transitioning from a fixed-length program to a variable-length program. Moreover, the adoption of an EPA framework will greatly influence the development of the program’s curriculum, teaching methods, and assessment strategies.^14,42

Finally, because EPA is a relatively new concept, the availability of African-based EPA studies is limited. Therefore, this study significantly enhances our understanding of how EPAs can be tailored to suit specific local contexts. Additionally, it plays a crucial role in defining the fundamental professional tasks that can be delegated to trainees for independent execution, especially in settings with limited resources.

Strengths and Limitation of the Study

One of the most notable strengths of this study is its meticulous approach, which involved conducting surveys and engaging expert panels to ensure a consensus was reached on the final set of EPAs. This nationwide consultative process ensures that the EPAs identified are not limited to a specific region or group, but rather represent a collective agreement from a diverse range of stakeholders. Additionally, the involvement of expert panels and surveys adds an extra layer of expertise and objectivity to the study. By involving professionals with specialized knowledge, the research benefits from their insights and ensures a well-rounded perspective. The study’s limitation could be that some respondents rated their level of agreement based on senior expert expectations rather than their own beliefs or experiences.

Conclusion

To reach consensus on the final set of 17 end-of-training EPAs, a meticulous approach was employed. As a result, these EPAs are valid, acceptable and representative of the discipline, and they can be used as a framework to inform Family planning and Reproductive Health Fellowship training program in Ethiopian medical education. This framework introduces a significant shift in the program’s structure, ultimately enhancing the quality of training and patient care.