Family Medicine Resident Experience Toward Workplace-Based Assessment Form in Improving Clinical Teaching: An Exploratory Qualitative Study

Introduction

Family medicine (FM) is known as one of the most important specialties worldwide due to the wide range of health services for all individuals regardless of age, gender, or bodily systems. It stands on a few important principles, namely, comprehensiveness, continuity, coordination, and accessibility.¹

In Saudi Arabia (KSA), FM was initiated in Riyadh in the early 1980s at the military hospital.^2,3 Initially, the Saudi Board of Family Medicine (SBFM) had a four-year-structured training program, which was then shortened to three years in 2019, covering wide aspects of knowledge, skills, and attitudes. The trainer and resident use different strategies to teach and learn various pieces of knowledge and skills throughout the program.⁴ Owing to the shorter duration of the training, the importance of using formative assessment is in high demand.

At the moment, the importance of clinical teaching and improving the use of assessment tools is gaining more focus compared to the previous years. In regard to teaching, FM residents in the clinic mainly present the challenges of providing appropriate patient care, maintaining efficiency, and incorporating meaningful education for residents under medical training,⁴ while the main goal of clinical training is to develop physicians with the capability of providing professional, effective, high-quality, and safe clinical care.⁵ For assessing the performance of residents in the clinic or any site of training, such as inpatient rotation, the formative tools of assessment are recommended. The most commonly employed tools are mini-clinical evaluation exercises (Mini-CEX) and case-based discussion (CBD). In the Mini-CEX, the resident during training is evaluated in multiple aspects, which include history taking, physical examination skills, communication skills, clinical judgment, professionalism, organization/efficiency, and overall clinical care.⁶

CBDs are known as the type of discussion that can introduce a resident under training to real-world scenarios where they can discuss their opinions and experience and interact with each other, trying to solve the problem in the case by searching for a solution together.^7,8 In the literature, Mini-CEX and CBD in a clinic or site teaching are useful for learning and improvement in addition to the known usage as formative assessments globally and locally. They have been used in different learning levels such as for undergraduate studies, internships, or postgraduate residency programs. It is shown that structured written feedback could improve the performance of medical students.^4,6,9

Moreover, it has been shown that using these forms can be affected by multiple cofactors in either usage or implementation such as a dual system of learning and assessment between a trainee and trainer, in addition to needs of time or variation in completions.^9–11 Reviewing articles show the need of providing support to the trainee to regulate learning in the clinical environment individually in respect of using tools and the support gained by supervisors in the training.⁹ Another study done for ER interns shows the costly significance to the emergency department due to using Mini-CEX encounters for interns’ performance assessment.¹⁰ In addition, the use of CBD for learning increases the knowledge and evidence usage of internal medicine residents.¹² However, locally in Saudi Arabia, Mini-CEX and CBD have been used by multiple residency programs as tools of assessment and learning, which shows that they are useful tools for assessing professionalism in medicine.¹³ They have been well structured in curriculums such as the family medicine program and anesthesia training program.^4,14

The introduction of a multidisciplinary approach early in the medical curriculum improves knowledge and skill acquisition. This is reflected in student performance, especially if evaluated using the Mini-CEX format, thus providing rapid feedback to students concerning their performance.¹⁵ In response to urgent requirements to expand FM services to achieve the transformative goals of the new Saudi Vision 2030, several curriculum design changes were recommended. A revealing recommendation was to reduce the residency training curriculum to three years, which, in turn, mandated a full review of the curriculum components and processes.⁴

From this point, the importance of attitude and perception of assessment and learning tools such as Mini-CEX and CBD is raised to ensure reaching the purpose of clinical training, which is to develop physicians with the capability of delivering professional, personal, effective, high-quality, and safe clinical care with intelligent kindness to provide care to the community in Saudi Arabia. For that, the question is raised regarding the attitude and perception of FM residents toward Mini-CEX and CBD forms for clinical teaching. This study aims to explore the experience of FM residents regarding the usage of mini-CEX and CBD as an assessment tool for improving clinical teaching and to determine the factors that can affect the use of WPBA among family medicine residents during training.

Materials and Methods

The current study is an exploratory qualitative phenomenology research targeting family medicine residents at the National Guard training center in KAH, Ministry of National Guard Health Affairs for exploring their experience of Mini-CEX and CBD toward clinical teaching improvement. This study protocol was approved by the Institute Research Board (IRB) at King Abdullah International Medical Research Center (Reference Number SP22R-235-10).

In this study, all 30 current residents were invited to participate. Residents were recruited using a purposeful sampling strategy and a total of 15 residents (7 from second year and 8 from third year) completed think-aloud cognitive interviews, sharing their thoughts on the overall impact of WBPA in the development of students’ clinical capabilities. Among the thirty residents, fifteen gave consent to participate, seven did not respond, six declined, and two were not contacted since the information had reached a saturation point. The collection of data was performed through the utilization of 1:1 interviews using semi-structured questions developed based on the literature review and expert input. The interview questions and probes are presented in the interview guide (Figure 1).

Figure 1 Data Collection tool-Interview Guide.

The selected participants were emailed an invitation along with a copy of the approved informed consent, which included consent for participation, the right to have the interview recorded, and the right to utilize the transcript responses for research purposes. Additionally, it was emphasized that the contributions would remain anonymous and be used only for research purposes. The interview guide with prompts and the confirmed date and time for the interview was emailed to the participants after obtaining their consent. Only the interviewer and interviewee were present during the interviews, which took place in a professional office at the workplace. In order to reduce the impact of the power dynamics between the interviewer and the participants and to establish norms and open communication throughout the interview, it was made clear during the introduction, ie, the purpose of the interview, the interviewer’s role in the study, assurances of confidentiality, and the freedom to participate or not without consequences or even to withdraw at any time. Each interview lasted for 30–50 minutes. Responses were collected until they reached the point of data saturation. As all participants were multilingual and capable of expressing themselves in the necessary amount of detail, all interviews were done in English using Microsoft Teams. Private and quiet settings were used for the interviews. The researcher gave the assurance that the interview would remain private and that only the research team would have access to the recordings after they were saved. After obtaining permission from the participants, the interviews were recorded and immediately transcribed verbatim by two independent transcribers to avoid any inaccuracy.

The analysis continued both during and after the data collection period. It took several readings and rereading of the interview transcripts to become completely comfortable with the data. The objectives of the study were reviewed frequently to regain focus when new themes and concepts developed. For analyzing the qualitative data, we used thematic analysis to highlight commonalities and patterns in the residents’ responses about their personal experiences through coding and categorizing themes. The interviews were transcribed using Microsoft Office. As an initial step, open coding to analyze individual interview data and researcher field notes was used to identify important words and label them accordingly. These open codes were created for each interview question and given a description in order to facilitate later coding and ensure that all data were relevant to the questions. Accordingly, data were examined using a line-by-line strategy to make certain that all data were appropriately labelled. Later, categories and themes were generated from these codes.

All coding was done by the relevant author, and it was confirmed by the other two authors. The updated analysis was delivered electronically to each team member for a second opinion. The study team then discussed and reviewed the findings during a weekly online discussion. On the final findings, the entire research team concurred.

Results

Fifteen participants were individually interviewed to reach data saturation (Table 1). The emerging themes were organized into the underlying principles of WPBA, the impact of the learning environment, associated opportunities and challenges, and making WPBA more effective (Figure 2). The emerging themes were supported by quotes from the residents’ interviews.

Table 1 Study Participants (N = 15)

Figure 2 Themes Generated from Interview.

The themes encourage trainers and residents should be more actively engaged in learning activities in their respective programs which helps them in remaining involved and enthusiastic about their learning pursuits. All the themes reflected the participants’ understanding of the concepts and the need to develop effective education and training programs to deliver quality healthcare and improve patient outcomes. The fundamentals of communication, opportunity to get timely feedback, clear guidelines, and curriculum update were emphasized by the participants. Arguably, results will enrich the learning environment by establishing the tools of assessment for feedback and better outcomes (See Table 2).

Table 2 “Inductive Analysis of Participants on Workplace-Based Assessment”

Theme One: Underlying Principles of WPBA

The study’s findings showed that every participant was well-informed regarding the presence of situated learning and the opportunities to receive structured feedback from an expert or a trainer.

A third-year male resident notes,

It means the assessment tool is used to assess the resident’s experience, attitude, and skill upon each rotation and how much the resident gains or benefits from the rotation.

All participants agreed that the assessment method ensures that all students should get timely feedback and enables residents to discuss and learn from their trainers. Their understanding can be evaluated in the context of their arguments during the interviews.

---it is a performance assessment in rotation so trainee get timely feedback and get chance to discuss with trainer

Also,

—because family medicine residents deal with many cases so there are benefit like discuss with senior consultants.

More specifically, the majority of participants connected assessment tool to the development of abilities.

it means the assessment tool used to assess the resident experience, attitude, and skill upon each rotation and how much the resident gain or benefit from the rotation.

They also identified that assessing skills in the training program is possible through the assessment tool being used in the training program and providing space for open discussion. Like one said:

I believe to ensure that resident is obtaining skills in related cases and discovering strength and weaknesses, the use of appropriate tool is very important.

Likewise, another resident said,

--value of WPBA come from the formulated discussion and proper documentation. Thus increase chance of discussion with trainer to get maximum benefit.

Theme Two: Impact of Learning Environment

Participants talked about how crucial a student’s environment is for learning. All participants agreed that an orientation program should include information that promote the growth of the necessary skills, and cognitive capacities. They opined that orientation from the program faculty was not adequate. However, senior peers were supportive and helped to create a good learning environment. For instance:

Yes, minimal orientation was given at the beginning of the residency regarding the format as requirement and needed numbers only.

Another participant said:

For example, “the orientation mainly came from the previous batches and senior colleagues’--‘they teach me how to us the form and how to send.

Aside from orientation content, the majority of participants emphasized that it is imperative that orientation programs should also explain the concept, purpose, and use of WPBA.

----they teach me how to use the form, purpose, and how to send.

Theme Three: Associated Opportunities and Challenges

Participants travelled through thoughts to look at ways in which the WPBA contribute to better learning outcomes. While explaining, many participants mentioned that situated learning, and communicating with the trainer, and receiving feedback were perceived as opportunities for WPBA implementation in clinical teaching. Whereas, barriers include time limits in preparation and discussion, considering the workload that can distract from the main rotation, and a lack of ideal implementation can minimize the value of WPBA.

A resident explained,

It can be used as a teaching tool in case of missing some points because it gives a real-time experience through repetition.

Additionally,

‘it can give clear ideas about the disease, the case and the treatment plan.

Residents reflected on the impact of WPBA in time management, communication with seniors, and lack of proper implementation. As second-year resident reported,

WPBA is taking time while at some time the trainer has no time to fill in the form.

In addition,

----load of such tools affects time management and is difficult to communicate with the senior.

----in some rotation cannot be applied properly because there are may not be a full knowledge of specialty or full opportunity to take history/examination and management plan.

Theme Four: Making WPBA More Effective

The participants openly addressed current WPBA practice and emphasized the modifications that are required to improve their knowledge and competence in dealing with residents’ skills building. The majority of participants placed a strong emphasis on updating the curriculum and addressing selection needs in addition to ensuring adequate electronic usage and e-source for WPBA.

In regard to the updated curriculum recommendation, a third-year resident shared,

preferred to be specified for each rotation and for each residency level by SCFHS in the curriculum” in addition to a second-year female resident who notes, “tailoring it (WBPA) according to the content of each rotation will ensure the proper implementation.

To make WPBA more effective, addressing selection needs was recommended, and it has been noted by senior and junior residents who participate in this study. Residents stated:

In each rotation, suggest or select reference trainers for family medicine specialists who are responsible for making an evaluation and overall evaluations.

The majority of participants—nearly half—believed that use of e-tools can improve the use of WPBA however, lack of orientation among trainers and technical problem has reduced its effectiveness. They explained it as:

We believe that to keep trainer oriented about the usage of e-tools, how to use them and post deadlines for more effective use of WPBA.

---also reduce technical problems of One45 because in most of the cases it doesn’t work such as filling the form as paperwork, finding the name of the trainer, and deadlines.

Discussion

This study aims to explore the experience of family medicine residents regarding the usage of WPBA as an assessment tool for improving clinical teaching in the Family Medicine Residency Training Program. In medical education, assessment serves as the guide at every stage of training. Evaluation is a crucial tool in assisting trainees in identifying areas for improvement and areas in which they are competent. It also helps residents acquire the abilities required to deal with a range of circumstances. Additionally, patient safety needs to be given first priority. Participating in evaluations gives recent medical graduates a unique opportunity to advance their careers based on feedback from Workplace-Based Evaluation assessors.^16,17

The participants in our study showed that they understood the importance of WPBA principles and that they had a shared understanding of them. This comprehension relates to the basic concepts and definitions associated with WPBA.^18,19

Updated curriculum and addressed selection needs were recommended by the participants in the study to reach the individual needs and the different rotation needs; this finding was consistent with the study showing that people who participate in both work and doing WPBA are gradually changing the workplace. Individual experiences and the perception of WPBA authenticity are framed by the environment and culture. In addition, different people tend to learn different things from the same experience.²⁰

Our result is consistent with previous studies that show the use of WPBA can be affected by multiple cofactors in either usage or implementation, assessment between trainee and trainer, in addition to needs of time or variation in completions.^9,11 With the presence of these barriers, the educational value of WPBA became low, and this finding is consistent with other studies that show the attitudes of the assessors sometimes reduce WPBA educational value.²¹

The effectiveness of workplace-based assessment is increased by several criteria, including the provision of feedback that is in line with the needs of the learner and concentrated on key elements of the performance. Faculty must also be properly trained for successful implementation because they play a crucial part.^20,22 Orientation programs should be introduced to residents and trainers with the support of curriculum designers.

The results of the qualitative analysis stressed that the healthcare system has a responsibility to access physicians who have undergone extensive training and proven to be competent in delivering safe and efficient patient care. Physician competence is always evolving during the educational process. Accurately evaluating a resident’s level of care and aligning it with a supervision level that supports both professional growth and safe care are essential. Authentic patient-care settings should yield trustworthy data that informs these judgments.²³

The current technique of a competency-based workplace assessment system provides data to guide judgments regarding a learner’s “Readiness to care for patients in an inpatient setting without the presence of a supervisor”. However, our residents perceived faculty buy-in and effort to be obstacles, particularly in programs lacking administrative support for assessment.²³

Nonetheless, this assessment system provides significant benefit to residency programs seeking to draw critical conclusions about their residents’ readiness for inpatient care in the absence of a supervisor. It is necessary to provide a model for other specialties in order to identify judgments, items, and instruments that address the progression of important abilities required to advance their learners along the continuum towards unsupervised practice.²⁴

Nonetheless, given that WPBA is a worldwide concern, the analysis’s findings are reliable. There are various limitations associated with this research. Due to the inherent generalizability restriction in qualitative investigations, there was limited generalizability. To address this issue, additional data is required in order to extrapolate the findings to different educational settings. Additionally, because this study was restricted to a single institute and a single specialty resident training program, particularly family medicine, this study was unaware of any additional programs or institutes. Further studies should be conducted on residents from various training programs in other training institutions may assess the skill sets of residents exposed to WPBA and investigate the practical implications of these assessment techniques on their clinical abilities. Furthermore, as this study focused only on the trainees, it did not examine the viewpoints of other stakeholders, such as the trainers. To address the mentioned limitations, we recommend that orientation programs should be introduced to residents and trainers with the support of curriculum designers. Moreover, stakeholders should be encouraged to improve implementation of WPBA and overcome associated barriers.

Conclusion and Recommendation

Experience with WPBA was explored with low satisfaction levels found among residents due to implementation-related issues. Experience toward WPBA was highlighted in improving clinical teaching but showed gaps between understanding and implementation.

There were multiple cofactors affecting ideal implementation such as time-related factors or a lack of time and workload, and because of the presence of these obstacles and barriers, the educational value of WPBA is minimal in senior residents compared to junior residents. The area of improvement would be mainly addressing appropriate prior preparation and utilization of available resources. Further studies are recommended, and stakeholders are encouraged to improve and fill the gaps. If training programs intend to equip residents with the skills needed for communities that rely on such residents, they should effectively implement the formative and summative assessment methods utilized in their program.