Readability Levels of Dental Patient Education Brochures
========================================================

* Catherine D. Boles
* Ying Liu
* Debra November-Rider

## Abstract

**Purpose:** The objective of this study was to evaluate dental patient education brochures produced since 2000 to determine if there is any change in the Flesch-Kincaid grade level readability.

**Methods:** A convenience sample of 36 brochures was obtained for analysis of the readability of the patient education material on multiple dental topics. Readability was measured using the Flesch-Kincaid Grade Level through Microsoft Word. Pearson's correlation was used to describe the relationship among the factors of interest. Backward model selection of multiple linear regression model was used to investigate the relationship between Flesch-Kincaid Grade level and a set of predictors included in this study.

**Results:** A convenience sample (n=36) of dental education brochures produced from 2000 to 2014 showed a mean Flesch-Kincaid reading grade level of 9.15. Weak to moderate correlations existed between word count and grade level (r=0.40) and characters count and grade level (r=0.46); strong correlations were found between grade level and average words per sentence (r=0.70), average characters per word (r=0.85) and Flesch Reading Ease (r=-0.98). Only 1 brochure out of the sample met the recommended sixth grade reading level (Flesch-Kincaid Grade Level 5.7). Overall, the Flesch-Kincaid Grade Level of all brochures was significantly higher than the recommended sixth grade reading level (p<0.0001).

**Conclusion:** The findings from this study demonstrated that there has generally been an improvement in the Flesch-Kincaid grade level readability of the brochures. However, the majority of the brochures analyzed are still testing above the recommended sixth grade reading level.

*   health literacy
*   oral health literacy
*   readability
*   Flesch-Kincaid grade level
*   patient education materials

## Introduction

Health literacy has been at the forefront of a national discussion and has been determined to be essential for improving not only the health of the nation as a whole, but individual health as well.1 Defined by the Institute of Medicine, health literacy is the degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.2-4 Understanding written materials (informed consent, patient education brochures and medication instructions) provided by health care providers is just one aspect of health literacy. According to the Institute of Medicane, “Health literacy is not simply the ability to read. It requires a complex group of reading, listening, analytical and decision-making skills, and the ability to apply these skills to health situations.”2 Patients not only need to be able to understand written materials, but also be able to communicate with health care providers adequately about their health care needs. Inadequate health literacy can not only act as a barrier for obtaining, comprehending and managing health related information, but can also act as an obstacle to accessing necessary health care.1

In the Surgeon's General Report Healthy People 2010, health literacy is identified as an important component of health communication, medical product safety and oral health.2 Efforts remain steady to educate and inform health care providers as to methods and strategies for improving health literacy to their patients. As evidenced in the most recent report, Healthy People 2020, Health Communication and Health Information Technology also address health literacy.5 The goal simply stated, is to use health communication strategies and health information technology (IT) to improve population health outcomes, health care quality and to achieve health equity. Objectives to reach this goal include: delivering accurate, accessible, and actionable health information that is targeted or tailored to a specific audience, increasing health literacy skills, and providing personalized self-management tools and resources.5

Literacy rates in the U.S. are staggering considering 24 million Americans (8.7%) are not proficient in English.6 In regards to health literacy, that number is even higher. In 2003, the National Center for Education Statistics conducted a nationally representative assessment of English literacy among American adults (age 16 and older) titled the National Assessment of Adult Literacy (NAAL). The NAAL health literacy levels were categorized into 4 performance levels determined by the U.S. Department of Education: Proficient, Intermediate, Basic and Below Basic (Figure 1).7 There is also a fifth level (Nonliterate in English) which includes adults who were unable to take the test because they could not speak English or Spanish. This study found 14% of adult Americans demonstrated “below” basic literacy levels. Regarding health literacy, the NAAL study found that only 25 million (12%) have proficient health literacy. The majority of the adults (53%) scored in the intermediate level and the remaining 77 million fell in either the basic or below basic literacy levels.8 This statistic indicates that approximately 47% of adult Americans have problems in understanding complex health information given to them by health care providers.1

There is a wide range of demographics that are affected by low health literacy. Older adults (65 years and older) were found to have the lowest health literacy scores when compared with other groups, with 23% falling below basic prose literacy range.7,9 Along with older adults, individuals with limited education, low English skills, low income, and those of ethnic or racial minority backgrounds are more commonly found to have lower health literacy.1 One-third of adults in the U.S. have difficulty reading and following through on health related information.10 Patients with limited health literacy reported having lower-quality communication with health professionals and confusion regarding medical terminology.6 Even individuals at the intermediate or proficient literacy levels can still have difficulty comprehending the “medical jargon” and the technical aspect of health information. Comprehension levels have been found to be about 2 or more grade levels below reading or education level, and when a person is under stress, the level drops even lower.11 What is more alarming is that patients with limited health literacy are less likely to use preventive services6 and have inaccurate knowledge about preventive measures such as water fluoridation, dental care visits and oral health-related quality of life.12

There is evidence to suggest that there is a strong correlation between a person's health literacy level and health outcomes.6 Specifically, those with limited health literacy are at a reduced capability to read labels and health messages, limits their ability to take medications, and lowers their likelihood of receiving preventive care and using emergency services instead. Studies also indicate that these individuals in turn have more hospitalizations and that among elderly people with limited health literacy skills, a poorer overall health status and higher mortality rates.6 The American Medical Association (AMA) also supports these findings through its report Health Literacy and Patient Safety: Help Patient Understand.4,5 It states, “Health literacy is a stronger predictor of person's health than age, income, employment status, education level, and race.”4

![Figure 1:](http://jdh.adha.org/https://jdh.adha.org/content/jdenthyg/90/1/28/F1.medium.gif)

[Figure 1:](http://jdh.adha.org/content/90/1/28/F1)

Figure 1: 
Levels of Literacy

Source: U.S. Department of Education, Institution of Education Sciences, National Center for Education Statistics, 2003 National Assessment of Adult Literacy

View this table:
[Table I:](http://jdh.adha.org/content/90/1/28/T1)

Table I: 
How U.S. Adults Obtain Health Care Information

Much of what we need to know or do regarding preventing, maintaining or improving our health is found in the written format.1,6 According to the 2003 NAAL report, most U.S. adults at the basic reading level obtained their health care information from these top 3 sources: radio or TV (92%), health care providers (89%) and family/friends (85%) (Table I). Yet, books or brochures (80%), magazines (79%) and newspapers (77%) were referenced almost as often as even health care providers (Table I). Thus, patients are obtaining their health care information from written materials or other sources as well as from their health care providers. Written patient education materials that are given to the patient may not be at an appropriate reading level and therefore should not be used alone for educating and or informing the patient. With patient education materials being distributed by health care providers, the readability of the documents should be looked at closely to determine if the patient can read, understand and retain this information for their health benefit. Regarding written patient education materials, the National Institutes of Health (NIH), the National Work Group on Cancer and Health, and the American Medical Association (AMA) all recommended the readability of patient education materials should be no higher than the sixth grade level.13

In 2000, a similar study was conducted by Alexander, and published in the Journal of the American Dental Association.14 The author selected 24 patient education materials from several sources and determined the Flesch-Kincaid readability level using a computer-based program. The conclusions from this study found that the readability was higher (41.7%) than the recommended reading level of seventh to ninth grade. The author concluded there needs to more attention on the preparation of patient education materials by making the documents easier to read and understood by the lay person.14

The purpose of this study was to evaluate dental patient education brochures produced since 2000 to determine if there has been any change in the readability levels of the brochures that are currently being distributed to dental patients.

## Methods and Materials

A convenience sample of 36 dental patient education brochures was obtained from private dental practices, a dental school and research facility to determine the readability level. The inclusion criteria were brochures produced between 2000 and 2014, from professional organizations American Dental Association (ADA), American Academy of Periodontology (AAP), American Academy of Pediatrics (AAPD), American Academy of Orthodontists (AAO), and only in English. Each brochures text was inputted into Microsoft Word to determine the readability using the Flesch-Kincaid grade formula. The Flesch-Kincaid formula calculates the reading grade level with the following formula: 

1.  The total words are divided by the total sentences and multiplied by 0.39

2.  The total syllables are divided by the total words and multiplied by 11.8

3.  The resulting numbers from steps 1 and 2 are added together

4.  Finally, 15.59 is subtracted from the resulting number of step 3

This formula was chosen since it is easily accessible to users and widely used on both PC and Mac computers as a built-in readability tool for Microsoft Office Word software. Other readability statistics were also calculated through this tool since it contributed to the overall readability of the document. These included: word count, characters count, paragraphs count, sentences count, average sentences per paragraph, average words per sentence, average characters per word, readability (passive sentences percentage), and readability Flesch Reading Ease. The Flesch Reading Ease formula calculates: 

1.  Average sentence length which is multiplied by 1.015

2.  Average number of syllables per word multiplied by 84.6

3.  Both these products are subtracted and the difference is subtracted from 206.835 to determine the reading ease of a document

The Flesch Reading Ease score correlates with an estimated reading grade level. The score index range is 0 to 100, the higher the score equates to text that is easier to read. Conversely, a score that is lower than 30 is considered to be at the college graduate reading level.1

Descriptive statistics were conducted based on the publications by professional organizations (ADA, AAP, AAPD, AAO) as well as the mean and standard deviation of the Flesch-Kincaid grade level for all 36 brochures collectively. Correlations between readability statistics were performed using the Pearson's correlation. A one-sample t-test was used to determine the Flesch-Kincaid grade level of all the brochures. Lastly, model building using a backward model selection was performed on these statistics to determine factors associated with the Flesch-Kincaid grade level. A p-value of <0.05 was considered significant. Data analysis was performed with the Statistical Package for the Social Science (SPSS version 22, IBM SPSS, Inc., Chicago, Ill).

## Results

A convenience sample of 36 dental education patient brochures was collected and analyzed (Table II). These brochures consisted of bifold, trifold and booklet designs. Topics included were diverse and consisted of specific information about diseases, conditions or procedures. All brochures included were produced by professional organizations with the breakdown of publications as follows: ADA (26), AAPD (3), AAP (4) and AAO (3). Descriptive statistics were performed for each professional organization publication set. Focusing on the Flesch-Kincaid grade level, the average mean grade level for total brochures was calculated as well as per professional organization. The average mean grade level for all 36 brochures was 9.15 with a standard deviation of 1.77. For each professional organization, the average mean grade level and standard deviations were: ADA – grade level 8.67 (SD 1.63), AAPD – grade level 8.90 (SD 1.05), AAP – grade level 11.30 (SD 1.70) and AAO – grade level 10.70 (SD 0.61). It is worth noting that the collective Flesch-Kincaid grade level readability of all brochures was significantly higher than the eighth grade reading level (p<0.0001).

View this table:
[Table II:](http://jdh.adha.org/content/90/1/28/T2)

Table II: 
Summary of Dental Patient Education Brochures

Pearson correlations were computed to determine the strength of relationship between pairs of readability statistics (word count, characters count, paragraphs count, sentences count, average sentences per paragraph, average words per sentence, average characters per word, readability - passive sentences percentage, and readability-Flesch Reading Ease). Weak to moderate correlations between word count and grade level (r=0.40) as well as with characters count and grade level (r=0.46) were noted. Strong correlations were found between grade level and average words per sentence (r=0.70), average characters per word (r=0.85) and Flesch Reading Ease (r=-0.98). Word count, characters count, paragraphs count and sentences count are highly correlated to each other, and the Pearson's correlation coefficients among them are from 0.86 to 0.996.

There were 4 groups of brochures (n=8) that were nearly identical in text but were produced in different years. A descriptive analysis showed that within the groups, the mean reading grade level reduction was between 0.8 and 1.4. The one exception being the brochure, Temporomandibular Disorders – TMD (ADA 2003, 2009), that had an increase in the Flesch-Kincaid reading level of almost an entire grade, 7.8 (2003) to 8.7 (2009). The brochure, “Periodontal Disease – Don't Wait Until it Hurts” (ADA, 2011, 2014) reported a Flesch-Kincaid reading grade level of 9.4 and 8.0, respectively. The brochure, “Do You Grind Your Teeth” (ADA, 2007, 2010) had a Flesch-Kincaid reading grade level reduction from 7.7 to 6.8, and the brochure, “Dental Implants – Are They an Option for You?” (ADA, 2011, 2014) reduced the Flesch-Kincaid grade reading level from 7.6 to 6.8. Within these 4 groups of brochures, the most current version in 3 of the 4 groups demonstrated a reduced grade level readability. Only 1 brochure, the 2014 ADA version of “Dental Implants – Are They an Option For You?” was able to reduce the reading grade level to the recommended sixth grade level (Flesch-Kincaid 6.8). Overall, the reading grade level of all the brochures collectively was found to be statistically significantly higher than the recommended sixth grade or below reading level (p<0.0001).

Lastly, model building using a backward model selection was performed to see what variables were associated with grade level. Average words per sentence (p≤0.0001) and Flesch reading ease (p<0.0001) were left in the final model. No multicollinearity was noted between these 2 factors with all variance inflation factors for each variable being less than 2.15 The adjusted R-squared is 0.996, which indicates the final model is adequate for this data.

## Discussion

The importance of health literacy and its relationship to an individual's health status has been documented in the literature.1,6 Health professionals including dental care providers still utilize written patient education materials as a method to inform and educate patients.16 With an estimated 90 million U.S. adults who have limited health literacy, 50 million are reading between the sixth and eighth grade level while the other 40 million have literacy skills scoring at or below the fifth grade level.2,4,7,13 It is imperative that the health care community be cognizant of the potential disparity between an individual's literacy level and their actual health literacy. Research has shown that written patient education materials are still a common tool given to patients as a means of informing and educating the patient. Health care providers including members of the dental team need to take into account that the patient education materials they may give to a patient describing a condition or procedure may be above their health literacy reading level and therefore should not be solely relied on for adequately educating and informing the patient.

A study conducted in 2005 evaluating the reading level of patient education materials from various health journals found that 50% of the sample had a reading level of eighth grade determined by the Flesch-Kincaid readability formula from Microsoft Word.17 Alexander's study from 2000 also found that the Flesch-Kincaid reading levels of the patient education materials that were evaluated were above the recommended reading grade level at that time.14 Both of these studies illustrated that not only is the Flesch-Kincaid computer formula often used for determining the readability level of a document, but also many of the patient education materials that are often distributed to patients may be above the recommended reading level.

Utilizing Microsoft Word Flesch-Kincaid computer formula, the data from this study showed that the mean range for reading grade level was between 8.67 to 11.30. Brochure topic areas of periodontics, orthodontics and tooth whitening had the highest reading level compared to brochures that were about implants, sealants, cracked teeth, the first dental visit or needing a bridge. Per publication, brochures produced by the AAP and the AAO had the highest readability level compared to those written by the ADA and the AAPD. There was a strong correlation between the Flesch Reading Ease and the Flesch-Kincaid grade level, but weak to moderate correlations with word and character count in regards to the reading grade level. Therefore, word and/or character count alone cannot be a predictor of overall reading ability. The use of Microsoft Word is one way to evaluate the readability of patient education brochures. It is considered an easy and convenient method, but Plain Language experts do not consider computer tests as a reliable tool used solely by themselves. The Flesch-Kincaid computer formula has been found to calculate lower readability measures compared to other computer tests and also when compared to calculating the readability by hand.

The findings from this study demonstrated that there has generally been an improvement in the readability level of the brochures that were sampled since Alexander's study in 2000. However, the majority of the brochures analyzed are still testing above the recommended sixth grade reading level.

### Limitations of This Study

The 36 brochures that were collected and analyzed were a convenience sample of what was available at the time of collection in the summer of 2014. Therefore, it is not reflective of every patient education material that is currently available. A larger sample size would increase the statistical power of this study. Another limitation is utilizing a computer program for determining the readability level of a document. The Flesch-Kincaid formula determines the readability of a text based on the average syllables per word and average words per sentence. It does not take into account the layout or design features of the documents which are other factors that can impact the overall readability level.

## Conclusion

Great strides are being made by professional dental organizations to reduce the readability level of written dental patient education brochures. However, the data from this research shows that there is still a need to continue to reduce the readability level so that written patient education materials will be more easily understood by a larger segment of the population.

## Footnotes

*   *Catherine D. Boles, RDH, MS, is an Assistant Professor in the Department of Periodontics at the University of Missouri-Kansas City, School of Dentistry. Ying Liu, PhD, is an Assistant Professor and Statistician in the Department of Biostatistics and Epidemiology in the College of Public at East Tennessee State University. Debra November-Rider, RDH, MS, is the Institutional Review Board Administrator at The Forsyth Institute and adjunct Assistant Professor at the Forsyth Dental Hygiene Program at MCPHS University.*

*   This study supports the NDHRA priority area, **Health Promotion/Disease Prevention:** Assess strategies for effective communication between the dental hygienist and client.

*   Copyright © 2016 The American Dental Hygienists’ Association

## References

1.  Wilson M. Readability and patient education materials used for low-income populations. Clin Nurse Spec. 2008;23(1):33-40. 
    
    

2.  Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services. Healthy People 2010. U.S. Department of Health and Human Services. 2001. 
    
    

3.  Health Literacy: A Prescription to End Confusion. Institute of Medicine. 2004. 
    
    

4.  National Network of Libraries of Medicine. Health Literacy. National Library of Medicine. 2014. 
    
    

5.  Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services. Healthy People 2020. U.S. Department of Health and Human Services. 2010. 
    
    

6.  Koh HK, Berwick DM, Clancy CM, et al. New federal policy initiatives to boost health literacy can help the nation move beyond the cycle of costly ‘crisis care’. Health Affairs. 2012;31(2):434-443. 
    
    [Abstract/FREE Full Text](http://jdh.adha.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMxLzIvNDM0IjtzOjQ6ImF0b20iO3M6MjI6Ii9qZGVudGh5Zy85MC8xLzI4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.  Kutner M, Grenberg E, Jin Y, Paulsen C. The health literacy of America's adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006-843). U.S. Department of Education, National Center for Education Statistics. 2006. 
    
    

8.  Cutilli CC, Bennett IM. Understanding the health literacy of America results of the national assessment of adult literacy. Orthop Nurs. 2009;28(1):27-34. 
    
    [PubMed](http://jdh.adha.org/lookup/external-ref?access_num=19190475&link_type=MED&atom=%2Fjdenthyg%2F90%2F1%2F28.atom) 

9.  Schillinger D, Piette J, Grumbach K, et al. Closing the loop physician communication with diabetic patients who have low health literacy. Arch Intern Med. 2003;163(1):83-90. 
    
    [CrossRef](http://jdh.adha.org/lookup/external-ref?access_num=10.1001/archinte.163.1.83&link_type=DOI) 
    
    [PubMed](http://jdh.adha.org/lookup/external-ref?access_num=12523921&link_type=MED&atom=%2Fjdenthyg%2F90%2F1%2F28.atom) 
    
    [Web of Science](http://jdh.adha.org/lookup/external-ref?access_num=000180378900010&link_type=ISI) 

10. U.S. Department of Health and Human Services. Simply Put – A guide for creating easy-to-understand materials. 2009. 
    
    

11. Barzel R. A Way with Words: Guidelines for Writing Oral Health Materials for Audiences with Limited Literacy. National Maternal and Child Oral Health Resource Center. 2008. 
    
    

12. IOM workshop examines oral health literacy. American Dental Association [Internet]. 2013 February 26 [cited 2014 October 12]. Available from: [http://www.ada.org/en/publications/ada-news/2013-archive/february/iom-workshop-examines-oral-health-literacy](http://www.ada.org/en/publications/ada-news/2013-archive/february/iom-workshop-examines-oral-health-literacy) 
    
    

13. Sabharwal S, Badarudeen S, Kunju SU. Readability of Online Patient Education Materials from the AAOS Web Site. Clin Orthop Relat Res. 2008;466:1245-1250. 
    
    [CrossRef](http://jdh.adha.org/lookup/external-ref?access_num=10.1007/s11999-008-0193-8&link_type=DOI) 
    
    [PubMed](http://jdh.adha.org/lookup/external-ref?access_num=18324452&link_type=MED&atom=%2Fjdenthyg%2F90%2F1%2F28.atom) 
    
    [Web of Science](http://jdh.adha.org/lookup/external-ref?access_num=000254772200034&link_type=ISI) 

14. Alexander RE. Readability of published dental educational materials. J Am Dent Assoc. 2000;31:937-942. 
    
    

15. Kutner MH, Nachtsheim CJ, Neter J. Applied Linear Regression Models. 4th ed. McGraw-Hill Irwin. 2004. 
    
    

16. Rozier G, Horowitz AM, Podschun G. Dentist-patient communication techniques used in the United States. J Am Dent Assoc. 2011;142(5):518-530. 
    
    [Abstract/FREE Full Text](http://jdh.adha.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamFkYSI7czo1OiJyZXNpZCI7czo5OiIxNDIvNS81MTgiO3M6NDoiYXRvbSI7czoyMjoiL2pkZW50aHlnLzkwLzEvMjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education papers in health-related journals. J Community Health. 2005;30(3):213-219. 
    
    [CrossRef](http://jdh.adha.org/lookup/external-ref?access_num=10.1007/s10900-004-1959-x&link_type=DOI) 
    
    [PubMed](http://jdh.adha.org/lookup/external-ref?access_num=15847246&link_type=MED&atom=%2Fjdenthyg%2F90%2F1%2F28.atom) 
    
    [Web of Science](http://jdh.adha.org/lookup/external-ref?access_num=000228345100005&link_type=ISI)