Clear as Mud: Readability Scores in Cloacal Exstrophy Literature and Its Treatment

Introduction

Cloacal exstrophy (CE), the most extreme presentation of the exstrophy-epispadias complex (EEC), is a rare congenital malformation characterized by an abdominal wall defect with exstrophy of the hemibladders and extrusion of the hindgut. Additionally, this disease has a wide spectrum of coexisting features leading to its second name: the OEIS complex (Omphalocele, bladder Exstrophy, Imperforate anus, and Spinal defects). CE is the least common presentation of the exstrophy-epispadias complex (EEC) with an incidence of 1:200,000 live births.¹ Although historically a fatal condition, surgical management has advanced, transforming CE into a uniformly survivable disease.²

This unfamiliar and morbid disease necessitates caregivers to learn about its presentation, associations, and sequelae to better understand their child and enhance their shared decision making. With the advent of the internet, allowing access to large volumes of information from various sources, caregivers have increasingly sought online medical information relating to their children.³ Health literacy (HL), the ability of individuals to obtain, process, and comprehend health information to make informed decisions, is well recognized as a critical link in the modern health system.⁴ The effect that inadequate HL can have on health outcomes is well documented including increased hospitalizations and poor medication adherence.⁵ Unfortunately, the average American reads at the 8th grade level. Nearly one-fifth of adults are unable to comprehend texts written at a 4th grade level.^6,7 Accordingly, the National Institute of Health (NIH) and American Medical Association (AMA) recommend that all medical information materials should be written at the 6th grade reading level.^8,9 In this study, the readability of online webpages regarding CE and its treatment are examined. The authors hypothesize that most webpages are written above the recommended 6th grade reading level and thus poorly understood by a significant number of the general population.

Methods

The search terms “cloacal exstrophy” and “cloacal exstrophy treatment” were reviewed on the Google search engine (https://google.com) and the Bing search engine (https://bing.com). Incognito mode and the disabling of “cookies” were utilized to reduce inherent bias of the authors’ search results. The first 100 search results for each search were collected on July 20, 2022. Each website was classified into one of three categories: non-profit, commercial, and institutional. The exclusion criteria consisted of duplicate websites, videos, research/news articles, and/or paid advertisements. Websites requiring payment or log-in information, such as UpToDate, were also excluded from review. Webpages with less than 200 words of content were also excluded to ensure an accurate evaluation of readability.

The readability of each webpage was evaluated using each of four independent validated scoring systems: The Gunning-Fog index (GFI), SMOG grade (Simple Measure of Gobbledygook), Dale-Chall index (DCI), and the Flesch-Kincaid grade (FKG). All 4 readability scores have been used frequently in evaluating the comprehensibility of online medical resources. Each portion of text was scored using an online tool (readabilityformulas.com). The GFI and FKG assign readability scores based on average sentence length and amount of “complex” words containing ≥3 syllables.¹⁰ Like the GFI and FKG, the SMOG index derives its score from the number of words with greater than three syllables, but it does not take sentence length into account.¹⁰ The scores of the GFI, SMOG, and FKG correspond to the US grade level required to understand the text (eg a score of 9.1 is understood by a 9th grade student). The DCI measures sentence length and “difficult” words that are considered outside a layperson’s familiarity. A score of 6.0–6.9 corresponds to a 7th/8th grade level, 7.0–7.9 corresponds to a 9th/10th grade level, 8.0–8.9 corresponds to a 11th/12th grade level, and a score above 9 indicates a college level. Statistical analysis was performed using one-way analysis of variance (ANOVA). Alpha was set at 0.05 to determine statistical significance. Analysis was conducted using JMP version 16.0.0.

Results

Forty-seven unique webpages were analyzed from a total of 400 search results (Figure 1). The majority of webpages were institutional (n = 28, 59.6%), with the remaining classified as commercial (n = 10, 21.3%) and non-profit (n = 9, 19.1%) (Figure 2). The mean readability scores across all websites were a GFI of 14.6 (SD = 2.51, range: 10.5–23, variance = 6.28), which correlates to college sophomore reading level; SMOG score of 10.8 (SD = 1.83, range: 7.8–17.2, variance = 3.34), correlating to an 11th grade level; DCI of 9.3 (SD = 0.86, range: 7–11.8, variance = 0.74), correlating to the reading level of an average college student; and an FKG of 11.8 (SD = 2.40, range: 8.2–20.3, variance = 5.76), correlating to an 11th grade level. Webpages were further classified based on SMOG score into grade level groups of 7th–9th grade, 10th-11th, and 12th and college level (Figure 3).

Figure 1 Sample Selection and exclusionary criteria.

Figure 2 Webpages categorized by type.

Figure 3 Webpage samples by grade level group based on SMOG score readability formula.

Readability was then assessed across different website categories (Table 1). There were significant differences across all readability formulas with significantly greater readability scores amongst non-profit websites when compared to institutional and commercial webpages (GFI p = 0.012, SMOG p = 0.018, DCI p = 0.021, FKG p = 0.0093).

Table 1 Average Readability Scores Based on Website Classification

Discussion and Conclusion

Discussion

CE is an uncommon condition among the general population. Historically, parents relied on their medical team for all medical information regarding their child’s condition. In the age of the Internet, caregivers have a wide selection of online health webpages to utilize. Assessment of the efficacy of this online platform is of increased importance amongst providers to ensure patients and their families can best understand their condition. This obstacle is prevalent amongst all fields of medicine. Numerous studies have been published assessing the readability of patient education materials, with a staggering trend towards texts written above the recommended grade level.^11–22

Our online queries yielded an average readability level that greatly surpassed the recommended 6th grade reading level set by the NIH and the AMA. Among the four different readability formulas, the lowest average readability level was equivalent to an 11th grade reading level. These results align with the authors’ previous findings regarding the readability of classic bladder exstrophy patient education material.²³ Unfortunately, there were no webpages that achieved the goal 6th grade reading level. Two webpages (Urology Care Foundation and Cincinnati Children’s Hospital) were found to be at an 8th grade reading level.

When stratified by organization type, we found that non-profit websites had significantly higher average readability scores and thus were more difficult to comprehend. Non-profit webpages required an average of a college reading level across all formulas to understand the material. Perhaps, this is due to non-profit webpages placing emphasis on more complex topics like etiology, surgical treatments, and risks and complications. Although a few patterns have been published regarding the difficult readability of non-profit websites, when examining the multitude of readability analyses, no clear trend emerges.^11–22 The Urology Care Foundation, a non-profit webpage, stands out as one of the lowest scored pages and thus was easily comprehensible. In a similar study, Routh et al compared internet sources between common and uncommon pediatric urology topics and found that webpages on uncommon topics like exstrophy had inferior accuracy and completeness.²⁴ Nevertheless, this underscores the importance for health professionals to work closely with professional and charitable organizations to ensure that online health information is both accurate and comprehensible.

Google and Bing search engines were utilized for all searches, despite the fact that some patients utilize less common search engines like Yahoo. A study design of 400 total pages from the two most common search engines should provide the most likely pages that patients would encounter. A limitation of this approach is that Google and Bing search engines provide personalized and targeted results and can present variable search results for each user based on their geographic location and search history.²⁵ Although this effect can be reduced with the use of an incognito window and disabling of “cookies”, the effect cannot be completely removed. All of the four readability formulas identify similar features from the text including word complexity, syllables, and sentence length which represents another limitation of this manuscript.

Conclusion

Inadequate HL is associated with negative consequences such as increased hospitalizations and poor medication adherence and contributes to health disparities between racial groups. Although the average American reads at the 8th grade level, this study found that most web resources directed at caregivers of children with CE are written for at least an 11th grade-level reader. Furthermore, materials posted by non-profit organizations were written for even more advanced readers, with an average of college-level reading skills required to comprehend the material. As a result, many caregivers of children with CE are not equipped to understand high-quality information about their child’s condition available on the Internet.

Practice Implications

Given the complex nature of repair and post-operative follow-up of CE, as well as the time-limited nature of patient-physician interactions, it is important that caregivers have access to publicly available resources that are both high-quality and easy to understand. This study highlights the large gap between the reading comprehension level of the average caregiver and the readability level of the available patient education material related to CE. To ensure the best outcomes for patients and their caregivers, and to enhance caregiver understanding and satisfaction, hospitals and healthcare systems must develop complete, understandable education materials to help families affected by CE.