Evaluating the readability, quality and reliability of online patient education materials on chronic low back pain

INTRODUCTION

Low back pain (LBP) is defined as pain, stiffness or muscle tension localized between the lower gluteal fold and the costal border.¹ LBP has been one of the leading causes of disability for the past 30 years and leads to a major burden on healthcare services and causes productivity losses.² More than three-quarters of patients with acute LBP (aLBP) have a good prognosis in terms of disability and pain, and return to work in less than a month.^3,4 Itz et al.⁵ reported that one-third of individuals with LBP recovered within 3 months but two-thirds reported pain even after 1 year. In addition to social, physiological and genetic causes, high levels of disability and pain, and the presence of comorbid conditions, also play a role in the transformation of aLBP into chronic LBP (cLBP).⁶ Superficial heat application, massage, acupuncture, spinal manual therapy, pharmacological treatment and multidisciplinary rehabilitation programmes are some of the treatment modalities that can be used for LBP.⁷

The internet has recently become an important source of health-related information. Patients access medical information through internet-based patient education materials (PEMs), soothe their worries and fears, and save time.⁸ A study found that 9 of 10 adult Americans used the internet in 2018 and nearly 75% of them researched health-related issues.⁹ According to the National Institutes of Health, the US Department of Health and Human Services, and the American Medical Association, PEMs on the internet should be written at a sixth-grade level.^8,9

If the readability of online material on a website exceeds this threshold, it is likely to be difficult to read and comprehend for the typical reader. As a result, it is critical that health-related material on websites be appropriate for the reader and thoroughly assessed before being used. There is increasing use of health-related information from the internet but it is unclear if the information provided is accurate, reliable, of high quality and readable. For this reason, scientific studies have assessed the quality and readability of information for various diseases on the internet.^9,10 A study on the readability and quality of online information about aLBP reported that the quality of information was low and the content was difficult to read.¹¹

Patients who have knowledge about the causes, pathophysiology, treatment and prevention of a disease are better able to participate and comply during the disease prevention or treatment procedures. As with other diseases, the level of knowledge of patients about cLBP is low.¹² It is clear that the transfer of reliable, high quality and readable information on this subject to individuals will play an important role in preventing cLBP, or in easy administration of treatment. We aimed to evaluate websites providing PEMs on cLBP for their readability, quality and reliability. In addition, we also tried to determine website typologies that provided reliable information about cLBP.

METHODS

Our study was conducted with the approval of the NonInterventional Research Ethics Committee (6488-GOA 2021/20-63). Two independent authors (EO, VH) searched the keyword ‘Chronic low back pain’ on Google (www.google.com.tr) on 26 April 2022. In case of disagreements among the authors during the evaluation of the websites, the final decision was made by a third independent author (YE). Google search engine was used because based on data from December 2021 it led the sector with a market share of 86.2%.¹³

During the website search, cookies and the computer’s browser history were cleared to ensure that the search results were unaffected by prior access. In addition, the study was conducted by logging out of all Google accounts. Completing the search, the first 200 websites’ uniform resource locators (URLs) were recorded, following the methodology of similar research in the literature.^14,15 The top 10 websites on the first page were ranked as the most viewed websites.¹⁶ The study excluded websites with non-English content, websites without information about cLBP, websites that demand registration or subscription, repetitious websites, websites with video or audio recording content but no written content, and websites with journal articles. In addition, graphics, pictures, videos, tables, figures and list formats in the texts, all punctuation marks, URL websites, author information, addresses and phone numbers, as well as references were not included in the evaluation to avoid erroneous results.¹⁷

During the website evaluation, if an evaluation criterion could not be identified on the homepage, the three-click rule was used,¹⁸ which states that a website user should be able to find any information in three mouse clicks or less. Although this is not a ‘rule’, it is believed that if information cannot be found in three clicks, the users will be unable to complete their task and will leave the site.

RESULTS

Our study included 200 websites; 105 were eliminated because they did not match the inclusion criteria, and the remaining 95 were evaluated. Commercial (30.5%) and health portal (17.9%) websites were found to be the most common when compared according to their typologies.

On Google’s first page, there are ten search results. There was no statistically significant difference between the first 10 search results and the remaining search results when they were analyzed according to their typologies (p=0.12). There was no statistically significant difference between the readability values of the top 10 websites and the readability values of the remaining websites (FRES, GFOG, GFOG, CL, SMOG; p>0.05). A significant result (p=0.045) was obtained when the Alexa values of the first 10 sites were compared to the Alexa values of the remaining sites. The top 10 websites on the first page were more popular in terms of search, viewing and traffic. There was no significant difference between the presence of JAMA reliability (p=0.06), DISCERN quality (p=0.16), HONcode presence (p=0.12) or typologies (p=0.12) of the top 10 and the remaining 85 websites (Table I).

These 95 websites had a mean (SD) JAMA score of 1.97 (0.99), a DISCERN score of 40.65 (19.94) and an Alexa score of 720406.64 (2029785.31) which suggest low reliability (JAMA score <3) and poor quality (DISCERN score 32–48). About a quarter (28.4%) of the websites were highly reliable with a JAMA score of >3. The text content of the 95 evaluated websites had a mean (SD) FRES of 55.74 (13.57) (very difficult), and the mean GFOG was 12.76 (2.8) (very difficult). The mean (SD) FKGL and SMOG were 10.12 (2.7) and 9.46 (2.10) years of education, respectively, while the mean (SD) CL index was 10.28 (1.96) years and ARI index was 10.32 (3.0) years of education. The readability indices were compared by site typology, and the results indicated no significant difference (p>0.05). A statistically significant difference was found when the readability index averages of 95 websites were compared to the grade 6 reading level (p<0.003) (Table I).

When the top 10 websites were compared to the other websites using content analysis, there was no statistically significant difference (p>0.05). Only websites with diagnosis (p=0.023) and symptoms (p=0.007) content had a statistically significant difference in their contents according to typology when all 95 websites were assessed (Table II). Commercial websites frequently mentioned diagnosis and symptoms.

There was a significant difference in JAMA reliability scores (p<0.001), DISCERN quality scores (p=0.002), and HONcode (p=0.008) by site typology. This statistical difference can be explained by higher JAMA reliability scores and DISCERN quality scores in government and professional websites. These scores were found to be lower in commercial content websites. Only 32 (33.7%) of all sites had HONcode. The highest number of HONcode were found on health portals (11; Table III).

The FRES, FKGL, SMOG, GFOG, CL, ARI, LW readability formula averages, JAMA and DISCERN scores, and HONcode entities were analyzed with respect to the site rankings. A weak positive correlation was observed between the JAMA and DISCERN scores (r=0.852, p<0.001) and JAMA and HONcode entities (r=0.351, p<0.001). The readability indices had a correlation, however, there was none between the readability scores and the popularity and visibility analysis index (Alexa; Table IV).

DISCUSSION

We evaluated whether the internet-based PEM related to cLBP is reliable, high quality and readable. We also tried to determine which types of sites provide highly reliable and readable information. We planned to compare the 10 most visited sites on the first page of a Google search with other sites in terms of quality, reliability and readability. Finally, we evaluated the relationship between the readability of the sites and their quality and reliability. In our study, the readability level of the internet-based PEM on cLBP was found to be well above the 6th grade level recommended by the National Institutes of Health. Website content was found to have low reliability and poor quality. It was found that health-portal-sourced websites provided more reliable and high quality information, whereas commercial websites performed poorly in comparison. By looking at the correlation between JAMA, DISCERN and HONcodes, we found that reliable sites also provided high quality information.

The internet has become a resource not only for patients but also for healthcare providers. More than 70% of adults search for health information online and more than 30% try to diagnose a medical problem in themselves or someone they care for.¹⁰

Approximately 90% of young people and adults in the USA currently use the internet, and internet use is increasing day by day across all age groups.¹⁰

It is known that online health resources provide patients with information on healthier lifestyle choices and regular health check visits.³⁰ PEMs strengthen communication between patients and doctors, increase patient awareness, and encourage patient-centered care.³¹ By evaluating the readability of online PEMs, previous studies determined that the contents required higher reading ability than that of an average American adult, and it was reported that a more suitable language should be used in PEMs.³² Texts consisting of long and complex sentences can affect the reader’s confidence while the reader tries to gain medical information and cause them to give up on reading the text. According to the US Department of Education, the National Literacy Institute, 32 million American adults are illiterate and 68 million Americans have a reading level below that of fifth graders.^33,34 Considering the increase in health-related information acquisition from the internet, providing more readable information on websites will offer individuals a better opportunity to prevent diseases and quickly evaluate the diagnosis and treatment processes when they are sick.

In our study, websites belonging to commercial and health portal typologies were the most common websites in all search results. Based on 10 Google searches, of the websites listed on the first page of the results three websites each were created by commercial, government and non-profit institutions. When compared based on their typologies, there was no significant difference between the top 10 websites and the remaining websites. However, there was a significant difference between website typologies and reliability scores. This significant diffference is related to the high JAMA scores on websites from the health portal. There was no significant difference between the top 10 websites and the remaining websites in terms of reliability and quality. No significant difference was found between the readability indices of the information on the websites based on the website typologies. There was no significant difference in information readability indexes between the top 10 websites and the remaining websites, but a significant relationship was found between Alexa ranks. The websites that appear on the first page of Google search results got more visits, which naturally increased their Alexa ranks.

Websites created by commercial sources accounted for the largest number of websites. Consistent with our findings, there are studies on different subjects demonstrating a higher number of commercial websites in the literature.^35–37 It has been established that these sites only have financial goals, do not provide reliable information, and may mislead their users. In addition, we found that 3 of 10 websites on the first page of Google search results were created by commercial sources, consistent with the literature.³⁸ Internet users often visit the websites on the first page of Google results to access information. These websites, which are promoted by Google for financial reasons, may misinform their users. We believe that websites with reliable information that do not have any financial goals should be ranked higher by search engines.

We found the presence of HONcode in 32 (33.7%) of 95 websites. Arif et al.³⁷ found HONcodes in 17.9% of the websites in their study, whereas Grewal et al.³⁹ found HONcodes in 16% of the websites. Our findings are consistent with the literature in this aspect. We found a significant difference in HONcode presence in websites according to typology. This difference was caused by health portals. In the literature, Chumber et al.⁴⁰ similarly found that health portals contained more HONcodes. Many studies report that the presence of HONcode provides reliable and quality information.^35,38 In addition, we found websites with HONcode to have high DISCERN and JAMA scores. Accordingly, we can deduce that healthcare professionals can inform their patients to refer to websites with HONcodes when searching for information about cLBP on the internet.

We found the mean DISCERN score of the websites was found to be ‘poor’ at 40.65 (19.94). Similarly, Guo et al.⁹ examined internet-based materials in the field of failed back spinal surgery and also found a mean DISCERN score of 35.26 (11.45). Moreover, there are studies in the literature reporting higher DISCERN scores.^41,42 However, the use of academic websites or websites of scientific journals in these studies led to high DISCERN scores as well as high JAMA scores and difficult readability. Patients prefer more readable sources with less medical terminology when they try to obtain health-related information from the internet. In this sense, academic resources are tailored for use among health professionals and to contribute to science, rather than being used by patients.

We found no significant difference between the top 10 websites and other websites in terms of readability of information. Similar to our study, Bagcier et al.³⁸ and Kocyigit et al.⁴³ also found no significant difference between the two groups of websites in terms of readability of information. Higher readability of the information in the top 10 websites that are visited more often will help users to understand the information easily and quickly.

When the readability of all website typologies was compared, no statistically significant difference was found. The average readability results obtained in our study were found to be well above the 6th grade reading level recommended by the National Institute of Health.⁴¹ In another study, Hendrick et al.¹¹ excluded academic websites from their analyses and found that the websites reporting information on aLBP had a moderate readability level than that understandable by the public. However, studies involving academic websites reported worse readability results.^38,43 We would like to point out that with easier readability levels, the contents of a website can reach wider audiences, and the power of information can be presented more effectively by providing information with a readability level that matches that of the public.

We found no significant relationship between popularity (Alexa) and readability indexes (p>0.05), however, a significant difference was found between Alexa and website typologies (p=0.007). This difference is related to Alexa values on nonprofit sites.These findings are similar to those in the literature.^42,43

Non-profit institutions try to provide quality information to users without financial concerns and try to reach more users. We can explain the reasons for high visits by the fact that these websites have won the admiration of the audience. We found a significant difference when the top 10 websites were compared with other websites in terms of their Alexa values. Naturally, this significant difference can be explained by the fact that the top 10 websites are visited more often.

When the content of the websites was analyzed, it was determined that there were 77 (81.1%) websites with treatment-based content, followed by 69 (72.6%) websites with aetiology-based content. A statistically significant difference was found between website typologies and topics. Websites prepared by non-profit institutions and commercial websites mostly covered topics related to diagnosis, whereas websites prepared by commercial and professional institutions mostly covered topics related to symptoms. Bagcýer et al.³⁸ also reported that websites on myofascial pain mostly referred to treatment-related issues. Furthermore, physiotherapy-based content, including exercise programmes, was the most common treatment topic on these websites. Based on these results, it can be said that popular topics relevant to a particular disease find more space on websites.

Our study has limitations. We only searched for websites in English, used a single search engine, used only ‘cLBP’ as a search keyword, and only detected internet sites that used a single country’s data network. Although there is no consensus on which index is ideal for assessing the readability of internet-based PEMs, the indices we used are among the most widely used. The websites were targeting an education level considerably over the appropriate level, according to all of the metrics we analyzed.

We believe that when creating health-related websites for the public about cLBP, which is an important cause of disability, the language of the website should be checked according to the relevant readability indexes, the content on the website should have a readability level suitable to the average education level of the country or countries for which the information is intended, and the website should contain high quality and reliable information.