Content validity of the Geriatric Health Assessment Instrument

ABSTRACT

Objective  Assess the content validity of the Elderly Health Assessment Tool with low education.

Methods  The data collection instrument/questionnaire was prepared and submitted to an expert panel comprising four healthcare professionals experienced in research on epidemiology of aging. The experts were allowed to suggest item inclusion/exclusion and were asked to rate the ability of individual items in questionnaire blocks to encompass target dimensions as “not valid”, “somewhat valid” or “valid”, using an interval scale. Percent agreement and the Content Validity Index were used as measurements of inter-rater agreement; the minimum acceptable inter-rater agreement was set at 80%.

Results  The mean instrument percent agreement rate was 86%, ranging from 63 to 99%, and from 50 to 100% between and within blocks respectively. The Mean Content Validity Index score was 93.47%, ranging from 50 to 100% between individual items.

Conclusion  The instrument showed acceptable psychometric properties for application in geriatric populations with low levels of education. It enabled identifying diseases and assisted in choice of strategies related to health of the elderly.

Validation studies; Geriatric assessment; Aged; Aging; Public health

RESUMO

Objetivo  Avaliar a validade de conteúdo do Instrumento de Avaliação da Saúde do Idoso com baixa escolaridade.

Métodos  Após a elaboração do instrumento de coleta de dados, o questionário foi submetido à avaliação de um comitê de especialistas, formado por quatro profissionais da área da saúde com experiência em pesquisas da epidemiologia do envelhecimento. Os especialistas puderam sugerir questões a serem incluídas/excluídas do instrumento, e avaliar cada bloco do questionário, observando se as dimensões a serem avaliadas foram abrangidas pelos itens do instrumento, em escala intervalar, como “não válida”, “pouco válida” e “válida”. Como medidas para avaliar o grau de concordância do instrumento, foram utilizados o porcentual de concordância e o Índice de Validade de Conteúdo. Considerou-se uma taxa aceitável de concordância o valor de 80%.

Resultados  O instrumento apresentou taxa média de concordância de 86%, variando, dentre os blocos, de 63 a 99% e, dentre os itens, de 50 a 100%. O Índice de Validade de Conteúdo apresentou escore médio de 93,47% e, dentre os itens do questionário, de 50 a 100%.

Conclusão  O instrumento apresentou qualidades psicométricas aceitáveis para ser utilizado entre a população de idosos com baixa escolaridade, permitindo a identificação dos agravos e auxiliando na escolha de estratégias, no que diz respeito à saúde do idoso.

Estudos de validação; Avaliação geriátrica; Idoso; Envelhecimento; Saúde pública

INTRODUCTION

The elderly population has seen exponential growth over the last decades, in developed and developing countries alike. In Brazil, e.g. 650 thousand people enter this age group each year, and most of them show increased vulnerability to chronic diseases. The promotion of improved quality of life among the elderly involves a myriad of Health-related factors, such as functional capacity retention, autonomy, social interaction and self-satisfaction, which must be taken into account, not only from a healthcare perspective, but also as part of a disease prevention and integral approach to geriatric health initiative.(¹)

The need to recognize elderly people’s true requirements and needs has fostered research in Geriatric Medicine and Health, often aimed to enhance understanding and application of reliable assessment procedures specifically tailored to the elderly population.(²)

Questionnaires are among the most valuable data collection tools employed in research, particularly population-based investigation, and is a low-cost, user-friendly data collection alternative.(²,³) However, such instruments must have psychometric properties (validity and reproducibility) that ensure reliability of selected indicators.(⁴)

Instrument validity, including content, criterion and construct validity, is a key dimension and refers to the extent to which a given instrument actually measures what it claims to be measuring. Content validity investigation, in turn, is subjective in nature and is designed to confirm the ability of the approach proposed to reflect certain behaviors in a given sample, thereby determining whether proper item selection, a key step in the development of a new instrument, has been achieved.(⁵-⁸)

Good, accurate, geriatric health-specific multidimensional questionnaires are scarce, particularly in developing countries, where the elderly population is often quite peculiar. These particularities tend to stand out in less developed areas, such as the Brazilian Northeast, where access to goods and services (i.e., education, health, sanitation, transportation and leisure, among others) is limited.(⁹)

Therefore, if a more comprehensive and effective understanding of the geriatric patient is to be achieved, the reliability of indicators in health assessment instruments must be confirmed.(⁵)

OBJECTIVE

Assess the content validity of the Elderly Health Assessment Tool with low education.

METHODS

This study investigated the validity of the Geriatric Health Assessment Instrument (GHAI). The study was conducted in Jequié, Bahia, in October 2013. The city of Jequié is located 365km southwest of Salvador and it ranks 9^th in terms of population, in the State of Bahia, with approximately 161,528 inhabitants(¹⁰) and a medium (0.66) municipal Human Development Index (HDI).(¹¹)

Instrument construction

The GHAI is a multidimensional questionnaire comprising seven blocks of questions based on previously validated instruments (Appendix 1),(¹²-²⁷) and its purpose is to investigate different health-related issues in elderly patients participating in the “Projeto de monitoramento das condições de saúde de idosos” [Geriatric health monitoring project]; Research Ethics Committee approval number 613.364; CAAE: 22969013.0.0000.0055.

Block 1 (personal and sociodemographic data) comprises 16 questions addressing sex, age, marital state, schooling level, ethnicity/skin color, religion and monthly income. Block 2 (housing conditions) contains four questions addressing housing conditions (house, apartment, shelter, etc.), home ownership and the availability of running water and electricity.

Block 3 addresses general habits and was based on other questionnaires(¹²-¹⁸) inquiring about eating habits, alcohol and tobacco use and level of physical activity. This block contains 18 questions adapted from the model proposed by Fonseca et al.,(¹²) as well as the instrument used by Munaro.(¹³) Questions relating to eating habits in the latter instrument (Munaro)(¹³) were adapted from Nahas et al.(¹⁴) while the item referring to behavior change stages was extracted from the model proposed by Prochaska.(¹⁵) Questions in this block ask about the frequency of consumption of certain foods in one week (cured meats, industrialized products, deep-fried foods, butter, non-diet soda, sugar, vegetables and fruits). Things like the daily number of complete meals, daily volume of fluid intake, occurrence of digestive problems leading to decreased food intake or appetite loss over the last 12 months, and self-perceived health were investigated. Questions concerning alcohol and tobacco use also address the duration of such habits.

Block 3 questions relating to the level of physical activity were extracted from the International Physical Activity Questionnaire (IPAQ)(¹⁶) and the questionnaires proposed by Reichert(¹⁷) and Pitanga et al.(¹⁸) The questions ask about the time dedicated to physical activities in a typical week, including work- and household-related activities, transport and leisure. The total sitting time in a typical day and the barriers to the practice of physical activities are also investigated, among other issues.

Block 4 (functional capacity) was based on the Kartz(¹⁹) and Lawton et al.(²⁰) scales. Questions ask about difficulties in walking or running, climbing stairs, sitting for long periods of time, bending down, extending the arms, using the telephone, shopping, cooking, taking medications, managing finances, etc.

Questions included in Block 5 (health status) were extracted from the World Health Organization Quality of Life Instrument (WHOQOL).(²¹) In this block, patients are expected to self-assess their health status compared to the past 12 months and to other people of similar age. Current diseases, medications taken and health coverage status are also addressed.

Block 6 (mental health) questions were extracted from the following questionnaires: Memory Complaint Questionnaire (MAC-Q),(²²) Mini Mental State Examination (MMSE),(²³) Pfeffer Functional Activities Questionnaire,(²⁴) Brazil Old Age Schedule (BOAS),(²⁵) Self-Report Questionnaire (SRQ-20),(²⁶) and the short version of the Yesavage Geriatric Depression Scale (GDS-15).(²⁷) This block addresses temporal orientation, memory, attention level, calculation and language skills. Questions ask about situations potentially experienced in the last 30 days (headaches, difficulty thinking clearly, loss of appetite, etc.), degree of self-satisfaction and mood, among other things.

Finally, Block 7 (quality of life) comprises 52 questions extracted from the WHOQOL(²¹) questionnaire and inquires about thoughts, feelings and certain aspects related to elderly people’s quality of life.

Instrument Evaluation

Once completed, the questionnaire was submitted to an expert panel comprising four healthcare professionals (PhDs and PhD students) with a background in research on epidemiology of aging, and chosen on a convenience basis.

Selected professionals were invited to participate in questionnaire validation and content testing. A letter was then sent with instructions regarding the purpose of analyzing the extent to which questionnaire items reflect proposed concepts, the relevance of target objectives and the significance of geriatric health assessment. An Informed Consent Term was also provided.

Evaluators were requested to suggest question inclusion/exclusion and to rate the ability of individual items in questionnaire blocks to encompass target dimensions. An adapted interval scale Hyrkäs et al.(²⁸) was used for this purpose, as follows: not valid (zero to 3 points), somewhat valid (4 to 7 points) or valid (8 to 10 points).

Calculations employed in instrument analysis were based on scores attributed to individual questionnaire items using the aforementioned interval scale. Therefore, rater perception was quantified regardless of variable categorization (i.e., descriptive/non-descriptive). Percent agreement (% agreement = number of participants that fully agreed about a given item/number of participants x 100) and the Content Validity Index (CVI = number of valid answers/total number of answers) were used as measurements of agreement about GHAI content validity.

The minimum acceptable interrater agreement in this study was set at 80%. Valid answer scores employed in CVI calculation corresponded to the sum of agreement scores entered in items rated 4 to 10, bearing in mind the exclusion of items rated 0 to 3 and review of those rated 4 to 7.(²⁸,²⁹)

RESULTS

The appropriateness of items proposed in questionnaire blocks and their respective relevance to measurement of target attributes (i.e., whether the instrument was comprehensive enough to provide a representative sample of the target behavioral domain) was expressed as inter-rater percent agreement.

The GHAI in this study achieved a mean percent agreement score of 86%; percent agreement ranged from 63 to 99% (Table 1) and 50 to 100% (Figure 1) between and within blocks respectively.

Figure 1
Mean percent agreement score per item

Table 1
Item analysis and percent agreement per instrument block

Blocks	Items	Items excluded	Percent agreement
	(n)	(n)	(%)
1. Personal and sociodemographic data	16	1	95
2. Housing	4	1	63
3. General habits	51	5	94
4. Funcional capacity	19	0	99
5. Health status	11	0	82
6. Mental health	54	39	75
7. WHOQOL-OLD	52	4	83
Total	207	50	86

Inter-rater agreement in his study was therefore considered acceptable (i.e., greater than 80%).

The sum of items rated as relevant yielded a mean CVI score of 93.47, with individual item CVI scores ranging from 50 to 100% (Figure 2). Items rated as non-relevant or non-representative (CVI score <100%) were not considered.

Figure 2
Mean Content Validity Index score per item. Fifty questions were excluded from the questionnaire, as follows: Personal and sociodemographic data (Block 1) - 1 question; Housing (Block 2) – 1 question; General habits (Block 3) – 5 questions; Mental health (Block 6) – 39 questions; WHOQOL-OLD (Block 7) – 4 questions (sections 1, 2, 3, 6 and 7 respectively)

DISCUSSION

The major issue facing health care services today is the growing elderly population, whose morbidity profile is dominated by chronic diseases that often lead to functional limitations, with negative impacts on patient’s quality of life.(¹⁹,³⁰)

This study evaluated a multidimensional geriatric health assessment questionnaire GHAI addressing different geriatric health-related aspects. Satisfactory levels of validity were achieved. Geriatric health assessment instruments are thought to reflect geriatric health status and assist health service managers in making decisions about the best care to be delivered to the population.

Percent agreement in this study was greater than the minimum acceptable value of 80%. The 80% cutoff point was adopted due to the reduced number of evaluators involved, bearing in mind the minimum of five evaluators required to reach a 90% global agreement score.(⁴,³¹)

Critical analysis led to questionnaire improvement. Some questions were excluded to better adapt the instrument to the target population.

Instrument length (207 Items) was thought to be excessive; therefore, 50 questions rated as non-relevant or non-representative were excluded to facilitate instrument validation and interviewee understanding. Shorter questionnaires are vital for expanded use in public health settings, if contributions to improved geriatric health understanding and care are to be expected.

A study investigating psychometric qualities of psychological and social well-being indicators applied to an elderly population living in Chicago (Chang et al.)(³²) suggested shorter, straightforward questionnaires are better suited for population-based health surveys involving elderly people, particularly in low schooling level settings, as in this study.

Some limitations of this study, particularly the reduced number of evaluators, precluded a more robust analysis of the instrument proposed and prevented comparisons with parameters employed in previous trials.

Content validation is a necessary step in novel assessment instrument constructions. However, other psychometric strategies must be considered to overcome the limitations introduced by the subjective nature of expert panel-based approaches.(⁴,⁶,⁸,³³) Yet, despite shared experiences, potential individual differences may interfere with instrument sensitivity and ability to properly reflect the health status of all individuals in the target population.

CONCLUSION

The results of this study suggest that the proposed Geriatric Health Assessment Instrument contains acceptable psychometric indicators of content validity. Therefore, the instrument can be applied in geriatric health surveys targeting populations with low schooling levels, and is a standardized, user-friendly tool for improved geriatric health understanding and appropriate healthcare decision making.

Appendix 1. Instrumento de Avaliação de Saúde do Idoso

ACKNOWLEDGEMENTS

This research was funded by Fundação de Pesquisa do Estado da Bahia (FAPESB), as part of the Scientific Initiative Programme.

REFERENCES

• ¹ Veras R. Envelhecimento populacional contemporâneo: demandas, desafios e inovações. Rev Saude Publica. 2009;43(3):548-54. Review.
• ² Virtuoso Jr. JS, Santos CA, Ferreira AN, Tribess S. Propriedades psicométricas da Escala de Atividade Física adaptada para mulheres idosas (EAFI). Geriatria & Gerontologia. 2008;3(2):96-103.
• ³ Schmitter-Edgecombe M, Parsey C, Lamb R. Development and psychometric properties of the instrumental activities of daily living: compensation scale. Arch Clin Neuropsychol. 2014;29(8):776-92.
• ⁴ Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet. 2011;16(7):3061-8. Review.
• ⁵ Rodrigues RM. Validação da versão em português europeu de questionário de avaliação funcional multidimensional de idosos. Rev Panam Salud Publica. 2008;23(2):109-15.
• ⁶ de Lacerda TT, Magalhães LC, Rezende MB. Validade de conteúdo de questionários de coordenação motora para pais e professores. Rev Ter Ocup Univ São Paulo. 2007;18(2):63-77.
• ⁷ Ottati F, Noronha AP. Parâmetros psicométricos de instrumentos de interesse profissional. Estudo Pesqui Psicolo. 2003;3(2):22-31.
• ⁸ Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94-104.
• ⁹ Morais EP, Rodrigues RA, Gerhardt TE. Os idosos mais velhos no meio rural: realidade de vida e saúde de uma população do interior gaúcho. Texto Contexto Enferm. 2008;17(2):374-83.
• ¹⁰ Instituto Brasileiro de Geografia e Estatística (IBGE). Diretoria de Pesquisas. Coordenação de População e Indicadores Socias. Cidades [Internet]. Bahia (BA): IBGE; 2014 [citado 2015 Nov 6]. Disponível em: http://www.cidades.ibge.gov.br/xtras/perfil.php?lang=&codmun=291800
  » http://www.cidades.ibge.gov.br/xtras/perfil.php?lang=&codmun=291800
• ¹¹ Programa das Nações Unidas para o Desenvolvimento. Instituto de Pesquisa Econômica Aplicada. Fundação João Pinheiro. Atlas do Desenvolvimento Humano no Brasil 2013. Ranking - Bahia (2010) [Internet]. Bahia (BA): Atlas Brasil; 2010 [citado 2015 Nov 19]. Disponível em: http://www.atlasbrasil.org.br/2013/pt/ranking
  » http://www.atlasbrasil.org.br/2013/pt/ranking
• ¹² Fonseca MJ, Chor D, Valente JG. Hábitos alimentares entre funcionários de banco estatal: padrão de consumo alimentar. Cad Saude Publica. 1999;15(1):29-39.
• ¹³ Munaro HL. Efetividade de uma intervenção educacional de curta duração sobre a diminuição da prevalência de fatores de risco para doenças e agravos não-transmissíveis (dissertação). Florianópolis: Universidade Federal de Santa Catarina, Centro de Desportos. Programa de Pós-Graduação em Educação Física; 2007.
• ¹⁴ Nahas MV, Fonseca, AS. Estilo de Vida e hábitos de lazer dos trabalhadores da indústria catarinense (1999 – 2004). Relatório Geral/ realização SESI/SC. Florianópolis: SESI, 2004.
• ¹⁵ Prochaska JO, Marcus BH. The transtheoretical model: applications to exercise. In: Dishman RK, editor. Advances in exercise adherence. Champaign, IL: Human Kinetics, 1994;181-90.
• ¹⁶ Matsudo SM, Araújo T, Matsudo VR, Andrade D, Andrade E, Oliveira LC, et al. Questionário Internacional de Atividade Física (IPAQ): estudo de validade e reprodutibilidade no Brasil. Rev Bras Ativ Fis Saude. 2001;6(2):5-18.
• ¹⁷ Reichert FR. Barreiras à prática de atividades físicas: prevalência e fatores associados [tese]. Pelotas: Universidade Federal de Pelotas; 2004.
• ¹⁸ Pitanga FJ, Lessa I. Prevalência e fatores associados ao sedentarismo no lazer em adultos. Cad Saude Publica. 2005;21(3):870-7.
• ¹⁹ Duarte YA, de Andrade CL, Lebrão ML. O Índex de Katz na avaliação da funcionalidade dos idosos. Rev Esc Enferm USP. 2007;41(2):317-25. Review.
• ²⁰ Lawton MP, Brody EM. Assesment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179-86.
• ²¹ Fleck MP, Chachamovich E, Trentini CM. Projeto WHOQOL-OLD: método e resultados de grupos focais no Brasil. Rev Saude Publica. 2003;37(6):793-9.
• ²² Crook TH 3rd, Feher EP, Larrabee GJ. Assessment of memory complaint in age-associated memory impairment: the MAC-Q. Int Psychogeriatr. 1992;4(2):165-76.
• ²³ Bertolucci PH, Brucki SM, Campacci SR, Juliano Y. O mini-exame do estado mental em uma população geral. Arq Neuropsiquiatr. 1994;52(1):1-7.
• ²⁴ dos Santos AA, Pavarini SC. Funcionalidade de idosos com alterações cognitivas em diferentes contextos de vulnerabilidade social. Acta Paul Enferm. 2011;24(4):520-6.
• ²⁵ Veras R, Dutra S. Perfil do idoso brasileiro: questionário BOAS [Internet]. Rio de Janeiro: UnATI; 2008 [citado 2015 Abr 17]. Disponível em: http://www.crdeunati.uerj.br/liv_pdf/perfil.pdf
  » http://www.crdeunati.uerj.br/liv_pdf/perfil.pdf
• ²⁶ Mari JJ, Williams P. A validity study of a psychiatric screening questionnaire (SRQ-20) in primary care in the city of Sao Paulo. Br J Psychiatry. 1986;148:23-6.
• ²⁷ Almeida OP, Almeida SA. Confiabilidade da versão brasileira da Escala de Depressão Geriátrica (GDS) versão reduzida. Arq Neuropsiquiatr. 1999; 57(2B):421-6.
• ²⁸ Hyrkäs K, Appelqvist-Schmidlechner K, Oksa L. Validating an instrument for clinical supervision using an expert panel. Int J Nurs Stud. 2003;40(6):619-25.
• ²⁹ Topf M. Three estimates of interrater reability for nominal data. Nurs Res. 1986;35(4):253-5.
• ³⁰ Pinto RB, Bastos LC. Abordagem das pesquisas em epidemiologia aplicada à gerontologia no Brasil: revisão da literatura em periódicos, entre 1995 e 2005. Rev Bras Epidemiol. 2007;10(3):361-9.
• ³¹ Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382-5.
• ³² Chang ES, Beck T, Simon MA, Dong X. A psychometric assessment of the psychological and social well-being indicators in the PINE study. J Aging Health. 2014;26(7):1116-36.
• ³³ Sireci SG. The construct of content validity. Soc Indic Res. 1998;45:83-117.

Publication Dates

• Publication in this collection
  Apr-Jun 2016

History

• Received
  12 Aug 2015
• Accepted
  29 Dec 2015