Relationship Between Research Culture and Research Activity of Medical Doctors: A Survey and Audit

Introduction

Clinicians play a key role in shaping research agendas, generating research questions, and conducting research that enhances rapid translation of findings.¹ Studies have found that health services whose clinicians conduct more research tend to have lower mortality rates, greater organizational efficiency, better staff retention and higher patient and staff satisfaction.^2–4 Additionally, a key recommendation to improve the estimated 85% avoidable waste in health research is the increased involvement of health service-embedded clinicians in driving research agendas.^1,5

In Australia, building research capacity of clinicians embedded within healthcare services has been identified as a key priority for health research,⁶ and most recently as one of the 12 priorities for the $20 billion Medical Research Future Fund.⁷ However, with increasing pressure on public health systems to meet activity targets and provide services to a growing and aging population, finding the time, money and resources to conduct research is challenging. Medical doctors are the second-largest health profession in Australia,⁸ and have an important role in many types of health research from bench to bedside. However, only 7% of the medical workforce report active involvement in research, and it has been argued this is decreasing.⁹

A key facilitator of building research capacity is an in-depth understanding of context-specific barriers and facilitators, research culture, and the levels of and drivers for current research activity. In health services in particular, arguments have been made that traditional research output measures, such as publications, may need to be replaced or supplemented by process measures.¹⁰ However, there have been very few studies comparing different forms of measurement in this context.

In Australia, the Research Capacity and Culture (RCC) survey has frequently been used to understand the current level of research culture and engagement of a profession, most commonly in Allied Health,^11–13 but also in medical professions.^14–16 Studies using this tool have found differences in research culture between teams, and findings have been used to inform tailored development of research capacity building strategies.^11,13 One study in Allied Health also found that there was no association between a team’s RCC score and their research activity; however, publication and project outputs were low (0–4 publications and 0–8 new projects per team), which limited interpretation.¹³

In line with the increasing focus on improving health professional research capacity, there is a need to robustly measure and evaluate outcomes of research capacity building initiatives. There is currently little understanding of the relationship between validated measures of research culture and capacity, and traditional research activity and output measures.

Thus, this study aimed to:

• describe the research capacity and culture of medical doctors across a health service using the Research Capacity and Culture tool
• describe the research activity, in terms of publications and projects, of medical doctors across a health service
• determine whether “team” level research capacity and culture on the Research Capacity and Culture tool is significantly different between specialty groups, and whether this score is associated with actual research activity

Materials and Methods

This cross-sectional, observational study collected data using a survey and audit. Ethics approval was obtained from the Gold Coast Hospital and Health Service Human Research Ethics Committee (HREC/10/QGC/177).

Context

Gold Coast Health (GCH) is a publicly funded tertiary health service located in South-East Queensland, Australia. The service includes two hospital facilities of 750 and 403 beds each, as well as outpatient and community-based services. Research is not currently routinely included in medical role descriptions, and engagement in research is variable across the organisation. GCH’s 2019–2022 research strategy has a focus on growing clinician research capacity and establishing a sustainable research culture.¹⁷ Strategic incentives and support for research include an annual grant scheme; access to small-scale grants for conference presentations and open access journal fees; an annual research week; a centralised Clinical Trials Unit; and a Research Council and Research Subcommittee of the Board to oversee implementation of the research strategy.¹⁷ Considerable work has also been done to increase Allied Health research engagement in the organisation,¹⁸ with the RCC tool used across multiple years to measure improvements.^13,19 This current study aims to build on this success in the medical stream.

Sample

Convenience sampling was used for the RCC survey. All medical doctors employed by GCH were invited to participate. “Medical doctors” included all professions requiring a medical degree (MBBS, MD or equivalent), including physicians, surgeons, anesthetists, radiologists and others. Data for the audit were gathered from institutional records on publications and projects.

Survey Tool

The Research Capacity and Culture (RCC) tool is a validated questionnaire, which measures indicators of research capacity and culture across three levels – organization, team and individual.⁸ The tool includes 52 questions (items) on self-reported success or skill in research, including 18 at the organisational level, 19 at the team level and 15 at the individual level. Each item is scored on a scale 1–10, with 10 being the highest possible level of skill or success, as well as an “unsure” option. The RCC also asks respondents to select applicable barriers and motivators to participate in research from a list of 18 of each, and prompts them to add additional barriers/motivators if desired. Lastly, the tool includes a standard set of demographic questions.

Small modifications were made to the tool to better fit the target population, and additional demographic questions were included (eg participant’s facility within GCH). For this study, participants were asked to reflect on their Medical College specialty (eg cardiology, general surgery) when answering questions about their “Team”. This clarification was made as participants could have also interpreted “team” to mean their multispeciality or multidisciplinary team. The tool was administered via a secure online platform (Survey Monkey^®).

Procedures

Active promotion and recruitment for the survey was in two stages, – the first in January and February 2019, then a break whilst the organization transitioned to an integrated electronic medical record, and then a second promotion in July and August. Potential participants were provided with an electronic participant information sheet and gave voluntary informed consent prior to participation. The survey took respondents approximately 15 minutes to complete.

Two key audits were performed to collect data on the publications and new projects of doctors in the health service for the calendar years 2018 and 2019. Data was collected for both years as, due to the distributed timing of the survey recruitment, it was determined that both years of data would be relevant to doctors’ RCC responses. For the purposes of the audit, new projects were defined as projects, which obtained a health service governance approval to proceed within that year. Health service-maintained research governance approvals and publication databases were used to identify projects and publications involving medical doctors from the health service. Where necessary, original articles were accessed, or authors/investigators were contacted to clarify information. Information on the number of doctors and FTE status was extracted from internal institutional records.

Data Analysis

Quantitative survey data analysis was performed using Stata 15 (College Station, TX, USA). Survey and audit data were analysed descriptively, using frequencies, percentages, medians and interquartile ranges. “Unsure” responses on the RCC were not included in the analyses but the percentage of “unsure” responses is presented for each item. A one-way analysis of variance analysis with post hoc Scheffe tests was used to determine whether there were differences between specialty groups on the mean of all RCC item scores at the “team” level. Linear regression was used to investigate the relationships between these means and the publications and projects per full-time equivalent (FTE) for each specialty group. Analysis of open-ended survey responses was conducted using inductive qualitative content analysis,²⁰ in which core meanings were derived from the text and grouped into themes. This analysis was completed by a single researcher with experience in health-related qualitative research (CB). Themes were discussed with a subset of the team with qualitative expertise (CB, CN, SM) to reach consensus.

Results

Survey: Quantitative Results

In total, 225 participants consented to complete the survey. Of these, 96 incompletely answered the questions and 129 completed the entire survey. Five of these complete responses were excluded as 4 were not doctors (allied health/nursing/midwifery) and one was a duplicate response. In total, data from 124 survey responses were available for analysis, representing 10.1% of the health service’s estimated total medical workforce.

Participant characteristics are shown in Table 1. The sample had almost equal numbers of males (49.2%) and females (46.8%). Consultants were overrepresented in the sample, making up 72.5% of respondents, whereas they make up slightly less than half of doctors in the health service. Conversely, registrars (12.1% of respondents) and junior doctors (12.9%) were under-represented. Research was part of the role description of 34.7% of respondents, and was not for 42.7%, while almost a quarter (22.6%) of respondents were unsure.

Table 1 Demographic Information and Professional Qualifications

The median score for research capacity and culture on the RCC was 5 at the organization level, 5.5 at the team level, and 6 at the individual level. Higher scores on the 10-point scale indicate a more positive perception of research culture and capacity. There was a high rate of “unsure” responses for the organization (18.6% of all item responses) and team (14.7%) levels, and a low rate for individual level (2.8%). “Unsure” responses relating to external funding, applications for scholarships/degrees, mechanisms for monitoring research quality, consumer involvement, and research software made up over a fifth of responses at both the organisational and team level. At the organisational level only, ensuring the availability of career pathways and having a policy/plan for research development also resulted in over 20% “unsure” responses.

Median scores for each item at the organizational level ranged between 3 and 7 (Table 2). Key strengths of the organisation’s research culture were “promotes clinical practice based on evidence” (median=7), “engages external partners (eg universities) in research” (6), “supports the peer-reviewed publication of research” (6) and “has regular forums to present research findings” (6). Lowest success was reported for “ensures staff career pathways are available in research” (3) and “has funds, equipment or admin to support research” (4).

Table 2 Median Score for Organisation Level RCC Items, Arranged in Descending Order

Median scores for items at the team level ranged from 3 to 7 (Table 3). Key strengths of team-level research culture reflected the organizational level, including “conducts research activities which are relevant to practice” (7), “supports peer-reviewed publication of research” (7) and “supports a multidisciplinary approach to research” (7). Lowest success was reported for “has funds, equipment or admin to support research” (3) and “has incentives and support for research mentoring activities” (3).

Table 3 Median Scores for Team Level RCC Items, Arranged in Descending Order

Median scores for items at the individual level ranged from 3.5 to 7 (Table 4). Strengths in individual research success were related to evidence-based practice (EBP) skills, including “finding relevant literature” (7), “integrating research findings into practice” (7) and “critically reviewing the literature” (7). The lowest rated item was “securing research funding” (3.5).

Table 4 Median Scores for Individual Level RCC Items, Arranged in Descending Order

Table 5 shows the most commonly reported barriers and motivators by percent of total survey respondents. “Lack of time for doing research” and “Other work roles take priority” were the most commonly reported barriers, identified by 83.9% and 82.3% of respondents, respectively. In decreasing order of frequency, other common barriers were the “lack of funds for research” (57.3%), “desire for work/life balance” (45.2%), “lack of a coordinated approach to research” (45.2%), and “lack of skills for research” (45.2%). Only 4% of respondents reported “not interested in research” as a barrier. The most commonly reported motivators were “to develop skills” (70.2%), followed closely by “increased job satisfaction” (66.9%). Other common motivators included “problem identified that needs changing” (58.9%), “career advancement” (58.1%) and “to keep the brain stimulated” (57.3%).

Table 5 Reported Frequency of Personal Barriers and Motivators to Conducting Research, Arranged in Descending Order

Survey: Qualitative Results

Twenty-nine percent (n= 36) of respondents provided free text responses to the question “Do you have any final comments or suggestions about the survey or research in general?”. Responses were a mean of 34 words in length. The four main themes are summarised in Table 6. The most common theme (n=21) was expansion on barriers to doing research, which were mostly reflective of the quantitative results. A significant portion (n=16) described the tension for change,²¹ emphasising there was a need for improvement in research engagement, positing that the health service was behind others in this aspect, and outlining frustrations with a perceived lack of focus on and support for research in the service. Others (n=11) included suggestions for potential strategies for improving the research culture at GCH in their response. These included suggestions like dedicated time for research, research support staff, making it easier to link in with potential projects, and calls for improved research culture and planning in the health service. A small number of respondents (n=4) outlined their perceptions of the benefits of increased research engagement as justification for the need for change.

Table 6 Codes from Qualitative Analysis of Final Free Text Question

Project Audit

There were 266 research projects that received health service governance approval from January 2018 to December 2019. 74.1% (n=197) of these had a medical doctor from the health service as an investigator. Data was not collected on investigators who were not affiliated with the health service. For every 12.5 doctors employed in the health service, one project was commenced each year.

Table 7 displays further details on these 197 projects. The health service was the only site in 50.8% of the projects, the lead site of a multisite study in 6.6%, and nonlead site on a multisite study in 42.6%. Most (80.2%) projects involved doctors from a single specialty in the health service, while 19.8% involved doctors from multiple specialties (eg cardiology and rheumatology). A third of projects involved collaboration with other professional streams, such as nursing and Allied Health.

Table 7 Characteristics of Publications and Projects from Medical Doctors in 2018 and 2019

Publication Audit

There were 479 total publications that included health service staff between January 2018 and December 2019. More than two thirds (68.3%; n=327) of these publications had a medical doctor from the health service as an author (data was not collected about authors not employed in the health service). For every 7.5 doctors employed in the health service, one article was published each year.

Table 7 displays further details on these 327 publications. There were around 160 unique authors each year (approximately 13% of GCH doctors), and just over a quarter of these published in both years. In 2018, 72.6% published a single paper, 13.7% published two papers, and only 1.2% published over ten papers. These results were similar in 2019.

A medical doctor from the health service was first or last author in 53.2% of the publications, and 13.8% involved authors from another professional stream in the health service, such as nursing or Allied Health. The most common publication type was primary research (59.9%), followed by other article types like letters or opinions (20.2%) and case reports (12.8%).

Relationship Between Survey Results and Research Activity for Specialty Groups

“Team” level scores on the RCC were summarised into 7 broad specialty groups (Table 8), which were (in no particular order): emergency medicine; surgery and intensive care; anaesthetics; psychiatry; paediatrics, obstetrics and gynecology; and 2 physician groups split according to which division of the health service the specialty was placed. One response could not be categorised as a specialty group. Specialty groups were anonymized as Groups 1–7 in accordance with this study’s ethical approval.

Table 8 Mean Scores for Team Level RCC Items and Publication and Projects per FTE, Separated by Broad Specialty Group, Arranged in Descending Order

Differences were identified in the mean total-item RCC scores at the “team” level between specialty groups (P<0.0001). Group 5 had the highest mean score (7.8) which was greater than Group 6 (3.1, P<0.0001), Group 2 (4.0, P<0.0001), Group 3 (4.6, P=0.010), and Group 4 (5.2, P=0.036). Group 6 had the lowest mean score (3.1) and was lower than Group 7 (6.0, P=0.003) and Group 1 (6.4, P=0.006) as well as Group 5.

Linear regression showed a relationship between mean “team” level RCC score and both projects (Figure 1) and publications (Figure 2) per FTE. Each additional project or publication per FTE corresponded to an increase of 5.3 (95% CI 2.0–8.6; P=0.002) or 6.6 (95% CI 4.8–8.5; P<0.0001) in the mean “team” level RCC score, respectively. It was estimated that of the total variance in “team” level RCC score, 29.95% was associated with variance in publications per FTE and 7.91% in projects per FTE.

Figure 1 Relationship between Projects per FTE of teams and the teams’ mean RCC scores.

Figure 2 Relationship between Publications per FTE of teams and teams’ mean RCC scores.

Discussion

The research capacity and culture of medical doctors at individual, team and organisational levels was moderate, with medians of 6, 5.5, and 5, respectively. This level of research culture and capacity, and a pattern of team and individual RCC scores being higher than organisational-level scores, is reflected in similar studies including medical doctors in other Australian health services.^15,16

The pattern and spread of results between the highest- and lowest-rated items found in this study is also broadly reflected in the results of these other studies.^14,15 This includes the general trend of Evidence Based Practice-related items scoring more positively than pure research-related items, and items relating to funding, career pathways and incentives for research activity consistently scoring the lowest.²² This reflects progress towards the commonly cited goal that all clinicians should understand/use research, while fewer will participate in or lead it.²³ Lack of time and other work roles taking priority as the most common barriers to undertaking research are also consistent across the literature for clinicians within health services.^12–15

A 2017 study using the RCC with Allied Health professionals in the same health service returned scores consistently 1–3 points higher for the organisation level, with an overall median of 7.¹⁹ Reasons for this are unclear, as it is unlikely the organisation’s research culture has changed so significantly in that time. The Allied Health clinicians also scored their team-level success slightly higher, with a median of 6, and their individual capability lower, with a median of 5.¹⁹ Notably, both studies also found a high level of “unsure” responses, especially at the organization level, implying that the health service could improve communication and promotion of institutional research supports and initiatives.¹⁹

Consistent with the Allied Health study, differences were identified between specialty groups at the team level, with mean scores varying from 3.1 to 7.8. Other research has also shown that different teams within the same health service may have different barriers, motivations and levels of research capacity and current activity.^{11,13,19,24,25} Literature on research capacity building in health settings has found that team-based approaches are likely to be most effective, as they allow strategies to be tailored to specific needs.^11,24–26 Tools like the RCC are a useful way for health services to capture their teams’ current research engagement and needs.

This study found that medical doctors were the largest producers of research projects and publications within the health service. Over a third of projects involved collaborations with other professions like Allied Health and nursing, reflecting the importance of the multidisciplinary team in modern models of care.²⁷ Due to differences in how studies measure and report research activity,²⁸ it is difficult to compare research activity with other health services.^29,30 Available international literature on research outputs from medical clinicians usually focuses on specific specialties, often further limited to registrars and academic doctors, rather than a whole of health service measurements.^16,31–35 However, one self-report survey found that for every 12.8 Australian physiotherapists employed in tertiary facilities, one article was published per year, compared to 7.5 doctors in this study.³⁶

We identified an association between a specialty group’s research activity and their self-reported, subjective research culture and capacity. Publications demonstrated a stronger association than projects, likely because the two highest scoring groups on the RCC (1 and 5) had relatively few projects. Further investigation showed that one of these groups had a high proportion of multiphase, multisite and complex projects. This indicates that the number of projects may be a poorer indicator than the number of publications, as a simple project like a retrospective chart audit is counted equivalently to a multisite, multiphase interventional trial. The same can also be true of publications; however, complex projects will likely result in multiple publications, which helps offset this effect.

Previous research has called into question the utility of research activity and output measures, such as publications for measuring clinician research capacity. As producing research is not the core role of clinicians, an improvement in research culture is likely to have a significant time lag before a measurable increase in outputs is realized.^10,12,37,38 Some authors have argued that self-report measures of research capacity and culture should be combined with traditional research activity measures.^12,13 However, this is the first study to identify that the two types of measure are associated, adding further weight to this argument.

Limitations

Limitations to the study were the low (10.1%) response rate for the survey and the fact that the sample was not random. Other studies using the RCC have achieved both lower^14,15 and higher^12,13,16 response rates. Selection bias may also affect results, as it is likely that those interested in research were more likely to respond to the survey. This may also explain the overrepresentation of consultants compared to junior doctors. A poor response rate from junior doctors has been found in other surveys of research culture.²⁴ Due to this, the results may be more likely to reflect the opinions of senior doctors, and should be interpreted with caution for the junior doctor and registrar populations.

Future Directions

Locally, results from this study were fed back to key strategic groups at Gold Coast Health in February 2020; however, the Covid-19 pandemic, staffing changes, and an organisational restructure have slowed the translation of this work. Nevertheless, funding has been obtained for a Knowledge Translation study to identify and implement evidence-based strategies for increasing medical engagement in research. The findings of this current study are being used to inform this work, provide a baseline measure, and help localise and tailor potential strategies.

In terms of wider implications, the findings of this work may help inform the approach to research capacity building in similar settings. This study has demonstrated that research capacity and culture vary widely between teams within an organisation, and comparison with other studies shows that it varies between organisations. While this means that the results should not be directly applied to other settings, there are common patterns, and this work adds to the literature on the research capacity and culture of medical professionals^15,16 to provide a set of results for comparison. More importantly, however, there are few studies reporting data on health service-wide research activity, in contrast to the plethora of research describing the research outputs of universities.³⁹ As health services and clinicians are increasingly called upon to be producers, not just consumers, of research,^3,4 systematic study of their research activity is essential.

The link between RCC results and actual research activity of teams should also be further explored, especially in longitudinal research. It would be valuable to determine whether there is delay between improvements in a team’s scores on the RCC survey, and translation into increased research activity in the form of new projects. This would serve to delineate whether a self-report measure like the RCC can reflect improvements in research capacity and culture more rapidly, thus being a more sensitive measure to change than traditional research output measures.

Conclusion

This study reinforces the most significant challenge faced in supporting clinical research within a tertiary health context governed by activity-based funding allocations: providing medical staff time to engage in research activities. New approaches to address this in an increasingly constrained fiscal environment are needed. This study also demonstrated significant differences between team’s reported research activity and culture, indicating that research capacity building initiatives may need to be tailored to specialty groups. Objective activity measures, particularly publications, were shown to be associated with a team’s self-rated research capacity and culture. A combination of subjective process and objective activity outcome measures may therefore be complementary when measuring the impact of research capacity building initiatives. The results of the RCC in this study are intended to identify areas for improvement and provide a baseline for multi-faceted and tailored research capacity building programs in the health service.