Differences in Supervision on Peer Learning Wards: A Pilot Survey of the Supervisor’s Perspective

Introduction

A global increase in health-care workers is predicted by the WHO.¹ To meet this need, an increasing number of students are being trained for their future roles. This in turn puts pressure on clinicians, resulting in a lack of quality clinical placements.² Placing multiple students on wards simultaneously to address this problem has the potential to create stress for the supervisor and reduce quality of learning for the students.³

Learning in the workplace is theoretically grounded in the idea that learning takes place in a social context where interpersonal interactions allow an individual to integrate others’ experience in their own learning.⁴ Peer learning is used as a pedagogical framework in clinical settings with multiple students present simultaneously.⁵ Peer learning can be defined as “people of similar social groupings who are not professional teachers helping each other to learn and learning themselves by teaching.”⁶ As well as providing increased educational capacity through increased clinical placements, numerous studies have shown the benefits of peer learning for students^7,8 and patients.^9,10 While students’ perspectives on peer learning have been widely investigated,^{5,7,8,11–13} the supervisor’s perspective has rarely been addressed, perhaps in part due to the lack of clarity of their role in peer learning.

Peer learning from the supervisors perspective has been described as learning by doing, an interactive and collaborative process,¹⁴ increased responsibility for the student, and the importance of the learning environment.¹⁵ In Sweden, apart from using peer learning occasionally and informally in clinical contexts, many wards have been set up specifically for student nurses’ peer learning, where peer learning is used as a pedagogical framework and implemented consistently. We previously found that these student wards were student-centred, used peer learning, involved multiple professions in student education, and had physical adaptations for students.¹⁶ It is unknown whether our previous findings are transferrable to other wards that use peer learning as a pedagogical framework, or even to wards without a pedagogical framework. It is of course possible for individual supervisors to use peer learning regardless of the ward’s pedagogical framework. However, our previous study found that a united approach of all staff to create a supportive learning environment is necessary for peer learning to take place.¹⁷ Scheduling students simultaneously is not enough for learning interactions and often leads to an increased workload for a supervisor to take care of multiple students individually. Neither does using peer learning as a framework necessitate peer interactions at every opportunity but allows the flexibility to adapt according to clinical or educational situations. The effect of using peer learning as a pedagogical framework on the supervisor’s perception of the characteristics of the learning environment and satisfaction in the workplace is currently unknown.

Surveys that currently exist to investigate the characteristics of the clinical learning environment focus on nurses’^18,19 and other health-care professionals’²⁰ own learning or focus on the students’ perspective.^21,22 The aim of this study was to develop and pilot test a questionnaire to investigate the characteristics of wards and supervision from the supervisors’ perspective. The focus was on identifying the characteristics rather than evaluating the experiences as good or bad. The secondary aim was to compare wards that use peer learning as their pedagogical framework, referred in this study as peer learning wards (PLW), and non-peer learning wards (N-PLW) that do not.

Methods

Study Design

We developed and pilot tested a questionnaire for supervisors working at PLW and N-PLW. Questions were about the characteristics of the learning environment, supervision of student nurses, and satisfaction with the educational quality and their work overall.

Study Setting

This study was conducted on hospital wards in Stockholm, Sweden. Some wards have voluntarily decided to use peer learning as a pedagogical framework for student nurses,²³ while regular wards with no specific framework continue to exist. Similar adapted student wards are increasingly used internationally.^2,24–26 Wards with a pedagogical framework of inter-professional learning^27,28 were excluded from this study. Although peer learning was used in other professions, peer learning wards were only set up for student nurses.

Participants

Contact with hospital coordinators at four acute hospitals in Stockholm was established to reach ward managers. Convenience sampling was used to select wards where the ward manager responded to our invitation to participate. The inclusion criteria were adult inpatient wards with undergraduate student nurses present for more than 2 weeks. The exclusion criteria were wards with explicit other pedagogical frameworks than peer learning (such as interprofessional learning), emergency departments and intensive care. The ward managers were interviewed by telephone to assess whether the inclusion criteria were met and to group the wards into PLW and N-PLW. Wards were classed as PLW if the ward manager reported an explicit and consistent use of peer learning as a pedagogical framework for the students. There was no specific pedagogical framework in any of the N-PLW. Further questions were asked to verify that the wards were comparable in terms of students (eg number of students present and length of placement), patients (number and type of illnesses) and clinical specialty. All the wards who agreed to participate were included.

All supervisors in selected wards were invited to participate. The supervisor role in this context involved overseeing students’ daily activities, including feedback, reflection, and together with other supervisors on the ward contributing to the final assessment of the student for each clinical placement.

An online questionnaire was sent by e-mail to the participants. The survey was active for 7 weeks, with two e-mail reminders during the period. All answers were anonymous. Figure 1 shows a flow chart of the inclusion process.

Figure 1 Flow chart of the inclusion of participating hospitals, wards, and supervisors.

Questionnaire Development

We applied the AMEE 7-step process for development of a questionnaire.²⁹

Step 1: A literature review was conducted to define the construct and to investigate what is known about PLW and N-PLW characteristics.

Step 2: Observations on a PLW were previously conducted to explore different ways that supervisors viewed their role in student peer learning.¹⁶ Furthermore, interviews with 15 supervisors who worked in PLW were conducted, enabling the participants to talk about the construct in their own words, which guided the wording in the questionnaires.¹⁷

Step 3: Literature review, observation and interview data were synthesised to determine areas of interest to investigate. Satisfaction with workplace and education questions was added based on validated general performance and satisfaction questionnaires.^30,31

Step 4: An initial pool of questions and response options were developed and then discussed and refined among the authors. Where applicable, participants were instructed to answer in relation to specific events during their most recent shift, aiming to capture concrete events and activities. Other questions aimed to elucidate participants’ opinions.

Step 5: Expert validation was conducted by four content experts who had published articles in a similar field and/or work with a relevant topic. They provided written feedback to semi-structured questions on the web-based questionnaire. The aim was to assess individual items’ relevance to the construct, as well as representativeness, clarity, relevance, content validity and distribution. The approach was qualitative rather than quantitative, to enable more detailed feedback from a small number of experts. The questions’ answer alternatives, wording and ordering were changed at this stage.

Step 6: Cognitive interviews were conducted individually with five supervisors (with experience from PLW and N-PLW but not included in the participant pool) to collect evidence of response process validity. These supervisors received a web-based questionnaire and were interviewed using a combination of the think-aloud technique and immediate retrospective verbal probing. These questions addressed comprehension, retrieval, judgement, and selection. The responses were analysed using basic coding. Questions and answer alternatives that were misinterpreted were changed or removed.

Step 7: Pilot testing was conducted with members of the target population (supervisors of student nurses on hospital wards in Stockholm) who completed the web-based questionnaire. A summary of the questions is presented in Table 1. The questionnaire consisted of 36 questions of which the first seven were control variables. The subsequent 29 questions were a mixture of single questions and compound questions containing multiple sub-questions. There were also qualifying questions that determined relevance for the participant to answer specific sections of the questionnaire determined by their workplace.

Table 1 Summary of Questions in the Questionnaire Distributed for Pilot Testing

Data Analysis of the Survey

Advanced Statistics

Advanced statistical analysis using Multivariate discriminant analysis was performed using Orthogonal Projections to Latent Structures Discriminant Analysis (OPLS-DA). This was chosen because of the need for a robust statistical method that allowed analysis of wide data matrices (many variables compared to subjects), and also a method that can handle missing values.³² OPLS-DA has the advantage of being able to analyse all variables together, which allowed us to look for patterns in prediction and investigate relations between all variables in a single context. This means that by using OPLS-DA all variables could be included in the analysis, which is not possible with more traditional regression models.³³ We used the non-linear iterative partial least squares (NIPALS) algorithm that allows analysis of wide data matrices, ie, many variables (items) in comparison to number of subjects. OPLS-DA uses a decline in Q2 (predictive fraction), calculated by cross-validation, to determine number of independent (orthogonal) components to extract to avoid over-fit. The analyses computed the influence of every X–variable on ward characteristics in the model, so that each sub-question was analysed individually. This means that composite scores were not used in this advanced statistical analysis.

Variable Importance in Projection (VIP) of the discrimination between the wards were calculated for each variable. VIP is a weighted sum over all model dimensions of the contributions of the variable’s influence. A VIP with a value exceeding 1.0 with a confidence interval not including zero was considered to have influence in the projection.^34,35 The statistical software SIMCA P+, version 17.0.0.2453, Sartorius Stedim Data Analytics AB, MKS, Umeå, Sweden, was used for PCA and OPLS-DA. SIMCA deals with missing values by letting the NIPALS algorithm interpolate the missing point using a least squares fit that gives the missing data no influence on the model.

Basic Statistics and Data Management

The scores were shown by the Shapiro test to be not normally distributed for almost all questions. Therefore, non-parametric methods were applied for descriptive statistics. For each whole question medians were compared between PLW and N-PLW. Composite scores were created for questions comprising several related sub-questions, and for this purpose partially answered questions were completed by replacing the blanks by the median of the participant’s other answers for that question. Statistical significance was tested by performing a non-parametric Mann–Whitney U-test. Relevant variables’ correlation was tested using Spearman’s rank correlation. The significance level was set to p<0.05. No correction factor was used as the results of the advanced statistics and their triangulation with the basic statistics would reveal any false-positive results due to multiple tests of significance.

The content in the comments sections for each question was cross-referenced manually to match and, if possible, replace a missing answer. Numerical scores were allocated to the answer alternatives on a Likert-scale, such as: never (0), rarely (1), half the time (2), often (3), always (4), where attributes associated with the characteristics previously found on a student ward scoring highest (and reversed where the question was negatively posed). Checkbox answer alternatives were scored based on a linear score allocation.

Results

Participants

Forty-six complete responses were collected for the pilot testing, 34 participants from seven wards that use peer learning as a pedagogical framework (PLW) and 12 participants from wards that do not use peer learning as a pedagogical framework (N-PLW). The overall response rate was 55% (60% for PLW and 44% for N-PLW). There were no statistically significant differences between the two groups in their baseline characteristics. All wards had multiple student nurses present simultaneously, and 83% of participants reported supervising multiple student nurses simultaneously. A flow chart of inclusion of participants is presented in Figure 1.

Psychometric Evaluation of the Questionnaire

A manual check of the answers removed four invalid sub-questions. These were due to ceiling effects, mutually exclusive answer alternatives possible, comments expressing confusion whilst leaving the quantitative alternatives blank, and temporary pandemic adaptations that did not reflect their ordinary practices.

Internal validity was checked by calculating the correlation using non-parametric Spearman’s rank correlation coefficient between items that measured similar variables (p>0.05): There was a strong positive correlation between peer learning activities and perceived peer learning (Rho 0.64) and a strong positive correlation between the two outcome variables about satisfaction (Rho 0.71).

A multivariable principal component analysis (PCA) was performed to find out which latent components could explain the results of the questionnaire.^32,34 The PCA analysed each sub-question separately, including the baseline characteristics, a total of 48 items. Two components explained 24% of the variance, using a cutoff of 0.15 in the loading score. The first component concerned the development of trust in the student’s competence and ability to integrate independently and explained 13% (Eigenvalue 5.92). The second was about trust in the supervisor’s own competence and abilities as a nurse, team player and supervisor of multiple students, and explained 11% of the variance (Eigenvalue 4.96). The PCA scores were uniformly distributed within the Hotelling’s T2 ellipse,³⁴ with one potential outlier, who had very little experience as a nurse. Cronbach’s alpha was 0.84 for the whole questionnaire, showing internal consistency and reliability of the item scores.

Difference Between PLW and N-PLW

Advanced Statistical Analysis

Discriminant analysis shows that the two groups answered differently, as shown, in Figure 2, by the horizontal axis that denotes the predictive component (score) of each individual answer to discriminate between the PLW and the N-PLW group (ANOVA p=0.0034). The spread of the answers in the vertical axis, which denotes systematic differences within each group that is orthogonal (ie independent to the predictive component), is much less in PLW than in N-PLW.

Figure 2 Orthogonal Projections to Latent Structures Discriminant Analysis plot (OPLS-DA), with circles representing the participants from the peer learning wards (PLW, black) and non-peer learning wards (N-PLW, white). Ellipse: Hotelling’s T2 (95%).

The VIP plot presented in Figure 3 demonstrates the variables that had the greatest influence on discrimination between the two groups. These were physical adaptations, various aspects related to peer learning, and student-led activities. These variables were also shown in the basic statistics to significantly differ between the two groups.

Figure 3 Variable Importance in Projection plot (VIP) with a value greater than 1.0. VIP value on the x-axis with confidence intervals in black.

Basic Statistics

After the overall analysis, we performed some basic statistical comparisons indicating that PLWs had statistically significant more student-led learning, student-centred questions, peer learning activities, perceived peer learning, physical adaptations for students, support for the supervisor, satisfaction with the education, and overall satisfaction. There were no near-peer interactions possible in any of the N-PLW because there were never students of different terms present simultaneously. There were no statistically significant differences in the other questions as shown in Table 2.

Table 2 Summary of Basic Statistics Comparing PLW and N-PLW from the Pilot Results

Discussion

A questionnaire to investigate the characteristics of wards and supervision from the supervisors’ perspective was developed and piloted to compare PLW and N-PLW. The questionnaire showed acceptable psychometric properties. The pilot results indicated, as expected, that PLWs had more peer learning activities and perceived use of peer learning than N-PLWs. In line with previous studies, the pilot results indicated that PLWs were more student-centred, had increased support for the supervisor and physical adaptations for the students.¹⁶ Supervisors on PLW reported higher satisfaction scores, and we discuss possible explanations for this.

The Questionnaire Can Be Used for a Larger Study

The questionnaire that we developed was designed to specifically study characteristics of individual wards according to the supervisor. The target population, supervisors of student nurses in inpatients, is the same as the population used for the pilot study. Conducting the questionnaire with a larger group could be used to investigate the effect of a particular pedagogical framework on the characteristics of a ward. The questionnaire could also be used to understand the relationship between the use of peer learning and the supervisors’ workplace environment and satisfaction.

As Expected, PLW and N-PLW Differed Most in Their Experiences of Peer Learning

Many of the characteristics that were most discriminatory between the wards were related to peer learning (Figure 3), both in the frequency of peer learning activities and perceptions related to stress, enjoyment and workload. PLW had statistically higher levels of peer learning as measured by frequency of peer learning activities and perceived levels of peer learning (Table 2). However, there was a lot of overlap, showing that peer learning is not always used even in a PLW and likewise that peer learning is used even in N-PLW. These findings are consistent with the view that peer learning as a pedagogical framework gives the opportunity to be flexible, giving insight into which situations call for a step away from PL.³⁶ This also implies that when multiple students are present it may be natural and intuitive to use peer learning even without a formal pedagogical framework.

Of note, all wards had multiple students present simultaneously, which was unexpected as we did not select for this. While some wards had extra supervisors present simultaneously to maintain individual supervision, most participants supervised multiple students. In N-PLW, there was no specific approach to the challenges this entails compared to supervising just one. This could be an indication of the decreasing opportunities to supervising one student at a time in future clinical settings. Thus, an emerging focus for research should change from exploring peer learning compared to individual learning, to exploring peer learning in the context of other approaches to supervising multiple students.

Activities Were More Student-Centred in PLW and More Supervisor-Centred in N-PLW

There are different views on the role of the supervisor in the clinical environment. We investigate the degree of student vs supervisor-centredness³⁷ used in the approach to clinical activities (students actively performing patient care, taking initiative) and learning activities (answering clinical questions). PLW supervisors had a more student-centred approach (Table 2), which is consistent with our previous study.¹⁷ The student-centredness has parallels with active learning, where the student is stimulated to deep learning through themselves engaging in acquiring knowledge and understanding.³⁸ The increase in students’ active learning, active question answering and taking initiative has also been found in student wards that use other pedagogical frameworks such as problem-based learning³⁹ and transformative learning,²⁸ which although different from peer learning has a similar model of the students focusing on the patient rather than their supervisor during clinical placements.

PLW Had More Investments in Students and Supervisors

Support for supervisors and physical adaptations such as a dedicated student room, computers, or clinical equipment, were among the most significant variables for influence of wards (Figure 3) and levels were higher in PLW (Table 2). This could be in part due to needing increased training for supervisors who are expected to adopt a specific educational model. However, another explanation could be that creating a united pedagogical approach to students, regardless of the specific framework, led to general investments in education.

Interestingly, there was no difference in the baseline level of training in education between participants in PLWs and N-PLWs. The contrastingly high level of informal and formal supervisor training available to participants on their current ward reflects that any recruitment bias (supervisors with high levels of supervisor training applying to or being accepted to work on a PLW more than an N-PLW) is unlikely to account for these differences.

PLW Supervisors Had Higher Levels of Satisfaction Than N-PLW

PLW scored higher than N-PLW on satisfaction scores (Table 2). To explore this further, we calculated correlations between these satisfaction outcome scores and other variables. This showed that the outcome scores for satisfaction with the educational quality, and overall satisfaction were not correlated to the extent of peer learning activities or perceived levels of peer learning. This implies that peer learning is not causative of the increased satisfaction, but rather the pedagogical framework of the ward. This is in line with our previous studies highlighting that the unified team approach to education is a key characteristic of student wards.¹⁶ Outcome scores were, however, positively correlated with other variables: extent of physical adaptations for students (educational satisfaction only); extent of support for the supervisor; and the supervisor viewing student supervision as important. Another explanation is that supervisors who consider their role as a student supervisor as an important part of their clinical role are more likely to choose their workplace based on the ward’s adaptations for students. Motivated supervisors could create a community of practice⁴⁰ centred around learning and shared goals among the staff members could contribute to increased satisfaction. This interpretation would be in line with our previous studies, which emphasise the role of a united approach to prioritise learning¹⁶ and supervisors upholding the community of practice¹⁷ as key components of wards that use peer learning.

Methodological Considerations

We followed a systematic questionnaire development guide designed for medical education and performed psychometric testing that demonstrated its validity. The questionnaire was piloted with the same target population as its intended future use. The pilot results fit with the theoretical framework that we used and are consistent with previously published results.

Bias

A limitation of this study may have been a selection bias towards wards with a higher rated work environment as ward managers who agree to participate are more likely to be interested in contributing to research. However, this selection bias could be assumed to equally affect PLWs and N-PLWs. More of the wards contacted were PLW than N-PLW, and the unevenness was accentuated by the higher response rate for PLW, perhaps reflecting greater interest in education and research.

Questions regarding specific situations during participants’ most recent shift were chosen to reduce selection bias of the scenario the participant answered about but could have introduced recall bias. A test–retest analysis could have been performed to increase the reliability. Participant’s reported activities could differ from their actual activities, and reporting bias could be greater in PLW supervisors trying to align their behaviour to the desired practices on the ward.

Confounders

The classification of PLW and N-PLW was based on the reported use of peer learning as a pedagogical framework by the ward manager. This included specially adapted “student wards” as well as wards with no special name. A limitation of this classification method could therefore be variation in the ways and extent of a peer learning framework implementation on the wards classed as PLW.

Statistical Methods

We started with an overall statistical analysis using the megavariate technique OPLS to be able to include all variables and thus avoid false-positive results. The significant variables in the OPLS discriminant analyses were then further analysed by Mann–Whitney U-tests showing that the same questions distinguished PLW and N-PLW.

The OPLS DA results should be interpreted as exploratory, and patterns of predictors and variable clusters could be interpreted from a bird´s eye view rather than focusing on each single predictor identified. Also, being an observational study, no conclusions can be drawn on causality. The triangulation of the results using descriptive and advanced statistics showed that the same questions distinguished PLW and N-PLW wards using both methods.

Future Directions

Before distribution of the questionnaire on a larger scale to the target population, modifications are needed based on the results of the pilot. Our results suggest that the presence of a pedagogical framework could account for differences in PLW and N-PLW. There are a lack of studies comparing PLW to wards with other pedagogical frameworks. Therefore, an emerging focus for research should change from exploring peer learning compared to individual learning, to exploring peer learning in the context of other approaches to supervising multiple students. An advanced statistical analysis of correlation between variables would further inform which variables correlate with the satisfaction scores. The questionnaire was designed for the nursing profession, but the questions are generically applicable to all health-care professionals with modification of specific questions according to the specific ward activities and learning aims. The questionnaire could therefore be modified for investigation of peer learning environments for other health-care students.

Conclusion

The questionnaire developed to investigate characteristics of wards and supervision on different wards was a valid and reliable instrument that can be used on a larger scale. Given the small number of participants in this pilot survey, the conclusions need to be cautious. Nevertheless, we found a significant difference between PLW and N-PLW in that they were more student-centred, had more support for the supervisor, more physical adaptations for students and higher levels of supervisor satisfaction, and as expected more peer learning. Whether the differences between the wards are related to peer learning specifically, or to the ward’s united pedagogical approach, is an interesting question for future investigation.