Psycometrics properties of subjetive indicator in children

Abstracts

OBJETIVO: Avaliar as propriedades psicométricas do Oral Heath Impact Profile (OHIP-14) para mensurar a condição de saúde bucal. MÉTODOS: Foram utilizados dados secundários de um inquérito epidemiológico realizado em 2001 pela Secretaria da Saúde do município de Sabará, MG, que consistiram de 312 formulários de exames da cárie dentária e de questionários de necessidade percebida (OHIP), referentes escolares de 12 anos, da rede pública. A confiabilidade do OHIP-14 foi avaliada utilizando os coeficientes alfa de Cronbach e correlação intraclasse. Para avaliação da validade de conteúdo foram realizadas a correlação de Pearson e regressão logística e para avaliar a validade de constructo foram realizados o teste "t" de Student e o de Tukey Kramer. RESULTADOS: A perda dos participantes foi baixa (entre 85 e 100%). A prevalência de cárie foi muito baixa sendo que 59,0% da amostra não apresentou cárie detectável. O modelo de regressão logística indicou valores de predição altos para declaração de necessidade de tratamento e médios para percepção de cárie dentária. A comparação entre diferentes escores médios do OHIP, de acordo com três categorias do índice de superfícies cariadas perdidas e obturadas (CPO-S), indicou diferença significante apenas para as categorias incapacidade social e deficiência. CONCLUSÕES: Os resultados indicam que o adolescente não relatou alta prevalência de doença bucal percebida. O OHIP está associado à necessidade percebida de tratamento, à declaração da presença de cárie dentária e ao aumento do índice CPO-S. Os resultados sugerem que o OHIP apresenta boas propriedades psicométricas quando aplicado em crianças, indicando ser um instrumento promissor para rastreamento de grupos prioritários para acesso ao serviço.

Saúde bucal; Condições de saúde; Levantamentos de saúde bucal; Índice CPO; Estudantes; Psicometria

ORIGINAL ARTICLES

Psycometrics properties of subjetive indicator in children

Christiane Alves Ferreira; Carlos Alfredo Loureiro; Vânia Eloísa Araújo

Centro de Pós-Graduação São Leopoldo Mandic. Campinas, SP, Brasil

^{Correspondence}

ABSTRACT

OBJECTIVE:To evaluate the psychometric properties of the Oral Heath Impact Profile (OHIP-14) to measure oral health in children.

METHODS: The OHIP-14 questionnaire was applied to 312 schoolchildren aged 12 years residing in Sabará, Brazil, in 2001. OHIP-14's reliability was evaluated using Cronbach's alpha and correlation intraclass coefficient. For examining content validity, Pearson's correlation and logistic regression analysis were applied. For construct validity, Student's t-test and Tukey-Kramer test were used.

RESULTS: Tooth loss among the subjects was low in this study (between 85% and 100%). Caries prevalence was very low and not detectable in 59.0% of the sample. The logistic regression model showed high predictive values for reporting treatment need and intermediate values for dental caries perception. The comparison between different OHIP average scores, according to the DMF-S index's three categories of decayed, missing, filled surfaces, indicated significant difference only for the categories social disability and deficit.

CONCLUSIONS: The study results indicated that the adolescents did not report high prevalence of oral conditions. The OHIP was associated with perceived treatment need, reporting of dental caries and increased DMF-S index. These results suggest that the OHIP has good psychometric properties when applied to children and could be a promising instrument for screening priority care groups.

Keywords: Oral health. Health conditions. Dental health surveys. DMF index. Students. Psychometrics.

INTRODUCTION

Indicators associating oral conditions to individuals' quality of life have been developed because of the need of understanding subjectively perceived health conditions as well as determining the impact of health conditions on quality of life.^6,19 The concept of perceived health need has arisen from the health promotion movement started in 1974, when the notion of health was dissociated from absence of disease. Health and disease have been then considered two distinct multidimensional entities and thus not measurable in a linear unidimensional scale. Health has been defined as a quality of life component.¹⁰

Among several instruments that have been developed to measure perceived needs, the Oral Health Impact Profile (OHIP) is most widely used in studies of people with different cultural background and sociodemographic profile. The OHIP was developed to provide a wide measurement of dysfunction, discomfort, and disability related to oral conditions. This information adds up to conventional indicators of oral epidemiology of clinical diseases providing thus a profile of "disease impact" on people and the effectiveness of health services in reducing these impacts. Originally, the OHIP comprises seven components and 49 items arranged in a questionnaire format and a 5-choice Lickert-like scale ranging from never to often is used in the answers. Data consolidation can be performed using a discrete scale or by scoring each scale interval from 0 to 4.¹⁸

Traditionally, perceived need indicators have been designed for adults and elderly. Still, Broder et al⁴ (2000) have conducted a study among urban teenagers of New Jersey, US, aged 12 to 14 years, comparing an objective indicator, DMF-T index (index comprising the sum of decayed, missed and filled teeth) and two subjective indicators (RAND-36-Item Health survey which measures general health similarly to the OHIP and comprises eight health concepts – physical health problems, role limitations caused by emotional or mental problems, energy/fatigue, emotional well-being, social functioning, pain, and general health perceptions – and the OHIP-49). The authors noted that subjective indicators, OHIP in particular, could be a valuable sensitive screening instrument for identifying those people in a community under high impact of oral health conditions.

The OHIP has been designed in English for a particular cultural background. Hence, for adapting the indicator into Portuguese and the Brazilian cultural background, Almeida et al¹ (2001) carried out a two-step cross-cultural translation. In the first step, the semantic validation of the cross-culturally translated instrument was carried out piloting it among users of the Brazilian Unified Health System's (SUS) health clinics from different age groups. In the second step, the instrument's psychometric properties were tested in samples of adult and elder populations.

Since OHIP semantic and psychometric validation had been carried out only in adult and elder populations and had not been before assessed in children and adolescents, this study aimed at evaluating OHIP psychometric properties in young people.

METHODS

Secondary data from an epidemiological survey conducted by the Sabará's Health Department in 2001¹ was used. The survey data had been collected through an informed decision process, researchers had contacted all participants' parents who authorized their children to be interviewed and the confidential report of findings. The utilization of the present study data has been previously authorized by the municipality's Dentistry Division and consisted of reports of oral caries examinations as well as questionnaires on perceived need (OHIP-14) applied to 312 public school boys and girls aged 12 years old from seven different areas. Other OHIP studies found variation coefficients (d/µ) ranging between 0.63 and 2.46 for dentate people and between 1.00 and 3.46 for edentulous people. In order to detect a 15% difference in mean scores for a category with a variation coefficient of 1.0, it was estimated that 251 people would be necessary for a type I error equal to 0.05 and a type II error equal to 0.20.

A simplified OHIP-14 questionnaire comprising 14 items was used as a measurement instrument. OHIP reliability was assessed using Cronbach's alpha coefficient and intraclass correlation coefficient (ICC). The ICC was used as an estimate of the total variability given by the sum of the natural variability and the measurement error variability.

Content validity was evaluated through two global scores: perception of treatment need and perception of dental caries. The correlation of OHIP with the global scores was assessed through Pearson's linear correlation. Univariate logistic regression analysis was applied to determine the association between the global scores, considered independent variables, and OHIP components.

To evaluate the construct's validity the sample was grouped according to the three categories of the DMF-S index (measuring the sum of decayed, missing, and filled surfaces): 0 to 5, 6 to 10, and above 10. DMF-S 0 to 5 was set as reference for allowing the comparing of the two other categories. A mean OHIP-14 was obtained for each one of the three groups. Means were then compared using Student's t-test for non-paired samples. Non-paired Student's t-test was applied to test the null hypothesis that OHIP means of the studied groups were equal.^2,3 The Tukey-Kramer test was also used for different size groups for allowing multiple comparisons (simultaneous inferences), particularly for all comparisons among DMF-S groups.

To assess item redundancy for each OHIP component a Kendall's non-parametric ordinal correlation analysis was carried out.

RESULTS

The study sample consisted mostly of females (160; 51.3%). When evaluated for a global subjective score, most (171; 55.5%) reported they needed oral treatment. Regarding their perception of dental caries, when asked, "do you think you have any cavities?", almost half were positive (124; 46.1%). This outcome had a high concordance with the occurrence of dental caries measured by both the DMF-S (45.2%) and DMF-T (41.01%) indexes.

Table 1 show the reliability assessment for OHIP's seven components.

The results of Cronbach's alpha coefficient were considered low ranging from 0.55 to 0.26. For all components together, a significant value of 0.8575 was found. Overall, ICC was similar to Cronbach's alpha.

Table 2 shows Pearson's correlation and regression analysis of three significant OHIP components and all components together.

The Pearson's test between OHIP components and reported perception of oral treatment revealed a positive though weak correction, ranging between 0.156 and 0.278. The difference between mean scores of each OHIP subscale, divided into those subjects reporting they needed treatment and those reporting did not need treatment, was significant only for the components pain, psychological discomfort, and psychological disability. The odds ratio ranged between 1.208 and 1.564, i.e., subjects who reported pain in the OHIP were about 1.5 times more likely to report treatment need when compared to subjects who did not report pain in the OHIP.

Data relating to each component were disposed into 2x2 tables where the lines show the two classifications of predictive variables, perception of treatment need and perception of having cavities and OHIP components. As the classifications were independent, odds ratio was calculated using the following equation:

OR = (a*d) / (b*c)

Odds ratio yielded point and interval values indicating the likelihood of a group of experiencing impacts when compared to other group (perception of treatment need and perception of having dental caries).

Table 3 shows the correlation and univariate regression analysis between each of the seven OHIP components and all of them together and reporting dental caries. The Pearson's linear correlation linear between OHIP components and the reported perception of dental caries showed a positive though weak correlation.

The difference between mean scores for each OHIP subscale, divided into those subjects reporting having dental caries and those reporting not having caries was significant for all components, except for deficit (p<0.06). The odds ratio ranged between 1.113 and 1.549. There were recorded two global perceptions regarding the subjects' oral health: need of oral treatment and perception of dental caries.

Table 4 exhibits the results of OHIP components according to the three DMF-S cut-off values: 0 to 5; 6 to 10, and above 10.

The comparison table among different mean OHIP scores according to DMF-S categories using ANOVA showed a significant difference for the components social disability and deficit.

Table 5 shows the results of the multiple comparisons.

The contrasts among DMF-S categories, arranged as dummy variables, indicated that, for the component social disability, the variable DMF-S 0 to 5 showed to be significantly different when compared to variable DMF-S above 10. For the component deficit, the variable DMF-S 0 to 5 had a significant difference when compared to the variable DMF-S above 10. All other comparisons were not significant.

DISCUSSION

The study sample consisted of 312 children aged 12 years from public schools in the municipality of Sabará, Brazil. The original study² with an active search had a high global response rate that varied per item studied. Regarding length of residency, missed answers were only 4.0% while for perception of treatment need 26.0% were either "do not know" or not answered. There was a higher response compared to 75% response rate reported for a questionnaire sent by mail by Slade et al¹³ (1995), 57.6% by Hunt et al⁷ (1995), and 74.4% by Slade et al¹⁵ (1996). These authors also noted that after reviewing several studies they found a response rate between 71% and 86% for the OHIP sent by mail. The average response rate was better than 86.8% reported by Miotto & Barcellos¹¹ (2001), and similar to 94.92% by Silva¹² (2001), both adopting the same search strategy as the present study.

OHIP-14 reliability was assessed through the internal consistency method of coefficient correlation using Cronbach's alpha coefficient. The measured reliability was low to moderate. Similar intraclass correlation coefficients (ICC) were obtained though they were systematically lower when compared to Cronbach's alpha coefficients. These results were lower to those previously reported by Slade & Spencer¹⁶ (1994) who obtained Cronbach's alpha coefficients ranging between 0.70 and 0.83 for six subscales and similar to 0.37 obtained in the deficit subscale. The present study showed lower results than those previously reported by Slade & Spencer¹⁷ (1994) that were between 0.42 and 0.77 for six subscales, and higher than 0.08 for social disability. These findings are lower than those reported by Locker & Slade⁹ (1993) who found values ranging between 0.80 and 0.90 for all components. They are also lower than 0.96 Cronbach's alpha coefficient obtained by Slade et al¹⁵ (1996) and 0.88 obtained by Slade¹⁹ (1997). Silva¹² (2001) described an internal consistency measured using Cronbach's alpha coefficient ranging between 0.6138 for psychological disability and 0.7720 for psychological discomfort assessed using the simple intraclass correlation coefficient. The obtained values ranged from 0.4427 for psychological disability to 0.6292 for psychological discomfort.

Some of the following factors could have contributed to the study results: method used, cross-cultural translation, fewer items in the questionnaire, and respondents' various schooling. Slade et al¹³ (1995) reported that when a standard interview was applied instead of a questionnaire sent by mail, OHIP-49 showed a lower internal consistency. Slade¹⁹ (1997) noted that in studies where the OHIP had been translated into other languages such as French and Spanish, some questions and categories could not be properly translated and thus affecting the instrument's reliability. In the OHIP-14's cross-cultural translation process, difficulties were reported concerning the translation of some terms into Portuguese.¹ Shortening the number of OHIP-49 items which gave rise to the OHIP-14 could have contributed to the instrument's lower reliability. In the OHIP-14 study, most result comparisons were carried out using the complete instrument. A general psychometry principle states that "the lower the number of items the lower the scale consistency". The OHIP-14 has a third less items than the original instrument, which could explain the low reliability seen in the present study.

Aiming at assessing whether there were potential redundancies in the questionnaire, internal correlations of OHIP items for each component were conducted. The Kendall's correlation showed concordance among the items, i.e., the OHIP-14 items captured different aspects including the contents of each component.

The Pearson's test between OHIP components and reported perception of oral treatment yielded a positive though very weak correlation. However, the difference between mean scores of each OHIP subscale, divided into subjects who reported treatment need and those reporting dental caries, was statistically significant for pain, psychological discomfort, and psychological disability. The odds ratio showed that subjects who reported pain in the OHIP were about 1.5 times more likely to report treatment need when compared to those who did not report pain. This finding suggests the good validity of OHIP's contents concerning the global indicator of reported perception of oral treatment need.

In the analysis of predictability using a logistic model, the likelihood of a subject to report pain as well as oral treatment need was high, suggesting a good validity of OHIP's content to predict the need for oral treatment. This finding shows that the OHIP has a good predictive value for reporting oral treatment need and the component pain had the highest predictive value.

The correlation analysis between OHIP components and the reported perception of dental caries also showed a positive though weak correlation. Despite that, the difference between mean scores and odds ratios of each OHIP subscale, divided into subjects who reported having dental caries and those who reported not having caries, was significant for all components.

The predictive logistic model applied to determine the effect of reporting in an OHIP component as well as reporting dental caries was moderate to low, indicating a moderate content validity of OHIP to predict the reported dental caries. These results indicate that, as expected for this age group, while reported pain had the highest predictive value of perception of dental caries, the predictive value of a positive reporting in the OHIP was only moderate for perceived caries.

To assess validity properties of the OHIP's construct the DMF-S index was divided into three categories: 0 to 5; 6 to 10; above 10. The comparison between different mean OHIP scores, according to the DMF-S categories and using ANOVA, showed a significant difference concerning social disability and deficit. In order to determine the categories in which differences were seen, the Tukey's test was used and a significant difference was found only in the ends of DMF-S for social disability and deficit. These results are similar to those reported by Slade & Spencer¹⁷ (1994), who found high impact in patients showing high values in objective indicators. They also corroborate what found Coates et al⁵ (1996) that HIV-positive patients who had many oral conditions reported a significant high impact. Moreover, they confirm the findings of Slade et al¹⁴ (1997) that oral conditions such as missing teeth, retained root fragments, and periodontal pockets among others were associated with high impact. It also partially corroborates Locker & Jokovic⁸ (1996) study when no significant association between subjective and objective indicators was found. Also, it agrees with Slade's¹⁹ (1997) research showing that higher OHIP scores were seen in individuals with less favorable oral-medical conditions (more missing teeth, more root fragments, more untreated caries, deeper periodontal pockets, and more periodontal recesses).

The statistical significance verified only in three components contrasts with the results reported by Broder et al⁴ (2000), who found a significant difference for all OHIP components while evaluating adolescents. Most probably this difference could be due the different epidemiological profile of the studied groups in Sabará and New Jersey. While in Sabará the mean DMF-S was 2.24 and 54.8% of the sample had DMF-S zero, in New Jersey the mean DMF-S was 8.8 and only 5% of the sample was DMF-S zero. As the frequency of individuals with DMF-S above 10 was very low in the present study, the statistical power of the sample was affected for rejecting the null hypothesis in some components. Overall, the frequencies found in the OHIP were much lower than those reported in other studies, including among adolescents.

The study results indicate that the prevalence of perceived dysfunction, discomfort and disability as an impact resulting of oral health conditions in children aged 12 years in Sabará, measured using the OHIP-14 instrument, was very low. The study results allow for reaching the conclusion that the OHIP instrument has good content validity for selecting groups of patients who perceive treatment need and only moderate validity for identifying patients reporting dental caries. The OHIP-14 has good criteria validity for the components social disability, deficit, and psychological disability when comparing the association of the instrument's scores with DMF-S index.

REFERENCES

Received on 9/1/2003

Reviewed on 30/9/2003

Approved on 4/9/2003

• 1. Almeida A, Loureiro CA, Araújo VE. Tradução transcultural do indicador de necessidade percebida perfil do impacto de saúde bucal (OHIP). Rev CROMG 2001 [
• 2. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.
• 3. Armitage P, Berry G. Statistical methods in medical research. 2nd ed. London: Blackwell; 1994.
• 4. Broder HL, Slade G, Caine R, Reisine S. Perceived impact of oral heath conditions among minority adolescents. J Public Health Dent 2000;3:189-92.
• 5. Coates E, Slade GD, Goss AN, Gorkic E. Oral conditions and their social impact among HIV. Aust Dent Serv 1996;41:33-6.
• 6. Cohen LK, Jago JD. Toward the formulation of sociodental indicators Int J Health Serv 1976;6:681-98.
• 7. Hunt RJ, Slade GD, Strauss RP. Differences between racial groups in the impact of oral disorders among older adults in North Carolina. J Public Health Dent 1995;55:205-9.
• 8. Locker D, Jokovic A. Using subjective oral health status indicators to screen for dental care needs in older adults. Commun Dent Oral Epidemiol 1996;24:398-402.
• 9. Locker D, Slade GD. Oral health and the quality of life among older adults: the oral health impact profile. J Can Dent Assoc 1993;59:830-8.
• 10. Locker D. Subjective indicators of oral health status. In: Slade GD, editor. Measuring oral health and quality of life. Chapel Hill: University of North Carolina. Dental Ecology 1997;15:3-7.
• 11. Miotto MHMB, Barcellos LA. Uma revisão sobre o indicador subjetivo de saúde bucal "Oral Health Impact Profile" OHIP. UFES Rev Odontol 2001;3:32-8.
• 12. Silva NS. Avaliação do efeito das condições sociodemográficas e utilização de serviços odontológicos sobre o perfil do impacto da saúde bucal sobre a qualidade de vida na população adulta idosa de Cuiabá-MT [dissertação de mestrado]. Campinas: Universidade Camilo Castelo Branco; 2001.
• 13. Slade GD, Brennan D, Spencer AJ. Methodological aspects of a computer-assisted telephone interview survey of oral health. Aust Dent J 1995;40:306-10.
• 14. Slade GD, Gansky SA, Spencer AJ. Two-year incidence of tooth loss among South Australians aged 60+ years. Commun Dent Oral Epidemiol 1997;25:429-37.
• 15. Slade GD, Hoskin GW, Spencer AJ. Trends and flutuations in the impact of oral conditions among older adults during a one year period. Commun Dent Oral Epidemiol 1996;24:317-21.
• 16. Slade GD, Spencer AJ. Development and evaluation of the oral health impact profile. Commun Dent Health 1994;11:3-11.
• 17. Slade GD, Spencer AJ. Social impact of oral conditions among older adults. Aust Dent J 1994;39:358-64.
• 18. Slade GD, Strauss RP, Atchinson KA, Kressin NR, Locker D, Reisine ST. Conference summary: assessing oral health outcomes-measuring health status and quality of life. Commun Dent Health 1998;15:3-7.
• 19. Slade GD. Derivation and validation of a short-form oral health impact profile. Commun Dent Oral Epidemiol 1997;25:284-90.

1

Technical report presented to the State Health Department of Sabará. Data to be published.

2

Data not yet published.

Publication Dates

• Publication in this collection
  08 July 2004
• Date of issue
  June 2004

History

• Accepted
  04 Sept 2003
• Reviewed
  30 Sept 2003
• Received
  09 Jan 2003