Development and validation of an instrument for measuring patient satisfaction with physical therapy

Abstracts

OBJECTIVE: To develop and validate an instrument for measuring patient satisfaction with physical therapy, and to identify the most important satisfaction indicators. METHOD: The sample was composed of 834 outpatients of both sexes with mean age of 46.7 years who were health plan users at 45 private health clinics in Natal and Mossoró, State of Rio Grande do Norte, Brazil, and who answered a questionnaire. The study covered the process of developing the instrument as well as the analysis of its psychometric properties. Qualitative analysis on the initial sample of items in the questionnaire was performed by a panel of specialists. A pilot study preceded refinement of the questionnaire and was carried out prior to data collection. Psychometric analysis (evaluation of the test that was developed) was performed by means of studying the reliability and validity of the measures obtained with the instrument. RESULTS: The results achieved in relation to reliability, by means of Cronbach's alpha coefficient (alpha = 0.94), and content, simultaneous, and construct validities showed high internal consistency and satisfactory validity according to psychometric standards for patient satisfaction with physical therapy. Factorial analysis indicated the existence of four dimensions in which patient-therapist interaction, especially regarding the physical therapist's communication skills, seem to be the best indicators of satisfaction. Aspects related to personal convenience, such as location of the clinic and availability of parking, were not strongly predictive of satisfaction with the care received. CONCLUSION: This study makes available a new tool to contribute to management and to the planning process necessary for improving the quality of physical therapy services.

patient satisfaction; reliability; validity

OBJETIVO: Desenvolver e validar um instrumento de medida da satisfação do paciente com o tratamento fisioterapêutico e também identificar seus principais indicadores. MÉTODO: A amostra foi composta por 834 pacientes ambulatoriais de ambos os sexos, com idade média de 46,7 anos e usuários de planos de saúde, que responderam o questionário em 45 clínicas da rede privada de saúde nos municípios de Natal e Mossoró/RN - Brasil. A pesquisa ocorreu desde o processo de desenvolvimento do instrumento até o estudo das suas propriedades psicométricas. A análise qualitativa da amostra inicial de itens do questionário ocorreu por meio de sua apreciação por especialistas. Um estudo piloto precedeu o refinamento do questionário previamente à coleta dos dados. A análise psicométrica (avaliação do teste desenvolvido) foi realizada por meio do estudo da confiabilidade e validade das medidas obtidas com o instrumento. RESULTADOS: Os resultados alcançados para a confiabilidade, por meio do Coeficient alfa de Cronbach (alfa = 0,94), e validades de conteúdo, simultânea e de construto, demonstraram elevada consistência interna e satisfatória validade segundo padrões psicométricos da satisfação do paciente com a fisioterapia. A análise fatorial indicou quatro dimensões em que a interação paciente-terapeuta, especialmente as questões relativas às habilidades de comunicação do fisioterapeuta, parecem ser os melhores indicadores de satisfação. Aspectos relativos à conveniência, como localização da clínica e disponibilidade do estacionamento, não indicaram forte predição de satisfação com a assistência recebida. CONCLUSÃO: O estudo disponibiliza nova ferramenta para auxiliar na gerência e no processo de planejamento necessários ao incremento da qualidade nos serviços de fisioterapia.

satisfação do paciente; confiabilidade; validade

SCIENTIFIC ARTICLES

Development and validation of an instrument for measuring patient satisfaction with physical therapy

Mendonça KMPP; Guerra RO

Physical Therapy Department, Universidade Federal do Rio Grande do Norte, Natal, RN - Brazil

Correspondence to

ABSTRACT

OBJECTIVE: To develop and validate an instrument for measuring patient satisfaction with physical therapy, and to identify the most important satisfaction indicators.

METHOD: The sample was composed of 834 outpatients of both sexes with mean age of 46.7 years who were health plan users at 45 private health clinics in Natal and Mossoró, State of Rio Grande do Norte, Brazil, and who answered a questionnaire. The study covered the process of developing the instrument as well as the analysis of its psychometric properties. Qualitative analysis on the initial sample of items in the questionnaire was performed by a panel of specialists. A pilot study preceded refinement of the questionnaire and was carried out prior to data collection. Psychometric analysis (evaluation of the test that was developed) was performed by means of studying the reliability and validity of the measures obtained with the instrument.

RESULTS: The results achieved in relation to reliability, by means of Cronbach's alpha coefficient (α = 0.94), and content, simultaneous, and construct validities showed high internal consistency and satisfactory validity according to psychometric standards for patient satisfaction with physical therapy. Factorial analysis indicated the existence of four dimensions in which patient-therapist interaction, especially regarding the physical therapist's communication skills, seem to be the best indicators of satisfaction. Aspects related to personal convenience, such as location of the clinic and availability of parking, were not strongly predictive of satisfaction with the care received.

CONCLUSION: This study makes available a new tool to contribute to management and to the planning process necessary for improving the quality of physical therapy services.

Key words: patient satisfaction; reliability; validity.

INTRODUCTION

The satisfaction of health service users has been increasingly considered as an index of the quality of care¹. Although the quality of care still constitutes a major concern in healthcare, the main focus of research has been the technical quality of the clinical specialties and not the aspects of patient satisfaction and opinion².

The definition of satisfaction described by Linder-Pelz³ is based on the sociopsychological theory which states that satisfaction is the expression of an attitude, i.e. an emotional response related to the belief that the service should present certain attributes (components/dimensions) and, thus, satisfaction is defined as the individual's positive evaluations of distinct dimensions of health service. This is a complex attribute and it is established according to the users' expectations⁴.

Redefining patients as health service consumers has stimulated their evaluation. Many of the available strategies to improve health services are based on the assessment of user satisfaction⁵. In health care, three components are considered important: technical assistance, interpersonal relationship and the physical environment. The evaluation of these aspects allows the estimation of quality of care. It is also necessary to take into account the specific characteristics of the country and/or region where the process takes place⁶. In the Brazilian cultural context, the patient's satisfaction with health care may be influenced by his/her sociodemographic characteristics, as described by several studies in the literature, especially in regard to gender, family income and educational level⁷.

There are many questionnaires available in the literature which measure the satisfaction with hospitalizations, medical visits or care provided by a health insurer. Physical therapy has features which can influence patient satisfaction. Intervention frequently demands much more time than a routine clinic visit. It involves physical contact, and therapy usually requires active patient participation. Therefore, an instrument to assess patient satisfaction with medical visits is not appropriate for physical therapy^8,9.

Several dimensions have been suggested in studies on patient satisfaction with physical therapy. These dimensions were classified as patient-therapist interaction^2,8; treatment, efficacy, convenience, comfort, overall satisfaction¹⁰; satisfaction, dissatisfaction, clinic location and costs¹¹; treatment, admission, logistics, overall satisfaction⁹; courtesy and privacy, admission efficiency, time of consultation/waiting time and convenience¹².

Therefore, in light of the absence of a consensus on which components are part of the construct and, also, in light of the lack of uniformity in the studies already conducted and expressed in the various methodological approaches described in the literature, we propose the development and testing of an instrument for measuring patient satisfaction with physical therapy, including the pertinent contextualization to Brazilian culture. We also propose the confirmation of the main indexes of satisfaction that the patient recognizes when forming an opinion about quality of care.

METHODS

The study on the development and validation of an instrument to measure patient satisfaction with physical therapy treatment began after approval by the Ethics in Research Committee of Universidade Federal do Rio Grande do Norte (Protocol Nº 077/02), warranting the consent of the respondents, the confidentiality and anonymity of the answers. The study was carried out as described below.

Theoretical basis of the test

This stage was initially based on a comprehensive literature review conducted by consultation of the numerous publications related to the patient satisfaction construct¹³. However, the main theoretical foundation of the new instrument's development was a previous study conducted by the same authors which was certainly the initial milestone for research in Brazil, with a rigorous psychometric analysis to validate instruments related to this construct, which focuses especially on the patient who receives physical therapy care¹².

Initial item sample development

According to Pasquali¹³, this stage must be conducted on the basis of a survey of the items contained in instruments which measure similar constructs. To the present date, five studies have been published on patient satisfaction and had their measures psychometrically tested^2,8-11. Furthermore, a list was prepared with questions which were considered relevant by the authors of the study and were present in other instruments that measure patient satisfaction with various health care specialties. A total of 40 items were listed.

Qualitative item analysis

The test content analysis was conducted through evaluation of its items in order to judge their relevance for the construct in question. A panel of experts was created with 35 physical therapists, including university lecturers, researchers, self-employed professionals, and managers. A minimal consensus by 80% of the experts was the criterion required to retain an item¹³. Five questions were considered inapplicable to the study population and others were adapted, at that moment, by the experts.

Participating clinics

The cities studied (Natal and Mossoró/RN-Brazil) had a total of 52 clinics belonging to the private health system and which provided physical therapy care to patients with private health insurance. Two of these clinics did not agree to take part in the study, and five others were not providing services to insured patients during the period of data collection.

Pilot study

The 35-item questionnaire was completed by 175 patients of 7 randomly selected clinics. To verify the cultural suitability of the instrument, all the items in the satisfaction scale included the option "I did not understand the question"^13,14. The authors determined that the items should have a percentage of incomprehension of less than 5%.

Refinement of measures

After evaluating the instrument during the pre-test, general aspects such as introduction, format and sequence of items were discussed. To refine the questionnaire, the following criteria were considered: Cronbach's alpha coefficient for each discarded item¹⁵; correlation between items of the instrument; redundancies; ambiguities and the percentage of incomprehension. Unanswered questions and/or crossed out answers received special attention.

Data collection instrument

The final version of the satisfaction scale consisted of 23 items which addressed aspects of various domains discussed in the literature on patient satisfaction (^{Appendix 1}). These items had to be answered on a five-point interval scale, ranging from "very poor" to "excellent" in the first 21 items. Similar scales were previously used in important studies on patient satisfaction^9,16-18. The last two items inquired about the future intentions of the patient in relation to the service. In these items, the scale ranged from "never" to "definitely".

Procedures

Subjects

Participants included male and female patients with private health insurance who received outpatient physical therapy care within the private health system of Natal and Mossoró/RN – Brazil.

Inclusion criteria

The subjects had to be at least 18 years old, be able to understand and answer the questionnaire, and have undergone a minimum of 5 and a maximum of 60 sessions of physical therapy treatment at the clinic where the data would be collected.

Data collection

The data were collected in the waiting room of 45 participating clinics (86.5% of the total). The first 20 patients with private health insurance, who arrived at the clinic for treatment and agreed to take part in the study, completed the instrument.

The clinics provided outpatient service in various physical therapy areas such as: orthopedic, rheumatic, neurological, and pulmonary conditions, among others. The data were collected between September and November 2003 and imported to the statistical software SPSS 11.0 for Windows for posterior analysis by the authors of the study.

Data analysis

Descriptive statistics, reliability and validity estimates were calculated for the instrument.

Reliability

Cronbach's alpha coefficient was used to assess the degree of internal consistency of the measures obtained. This general coefficient reflects the degree of covariance among the items, serving as an index of internal consistency of the instrument¹⁹.

Validity

Content, simultaneous and construct validations were conducted to assess the validity of the measures obtained with the instrument. These validations were used for psychometric analysis of instruments in recent studies on patient satisfaction with physical therapy^2,8,9,11.

Content validity

Content validation consisted of a systematic evaluation of the content of the instrument to assure its representativeness, i.e. to guarantee that the various aspects of the construct were covered by the questionnaire items²⁰.

Simultaneous validity

To determine the simultaneous validity of the instrument, the results were correlated to one criterion, that is, the direct and independent measure correlated to the behavior to be measured²⁰. Overall satisfaction measures were used as criterion variables (items 21, 22 and 23) ^2,8,21.

Construct validity

In order to evaluate the validity of the construct, multitrait-multimethod matrices were initially used to identify the consistency of the inter-item correlation of the measuring instrument.

The factorial structure of the satisfaction indexes was identified by submitting the data to a multivariate analysis known as factorial analysis. This analysis was conducted after the exclusion of 3 overall satisfaction items. These items were discarded because they are related to general evaluation of care, whereas the effort of the factorial analysis, in the present study, was to isolate specific dimensions of satisfaction¹¹.

Prior to the factorial analysis, the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) was used to determine whether the data fit the factorial model. The extraction method used was Principal Component Analysis^{15, 22}.

Initially, the criterion established was only the retention of factors with eingenvalue equal to or greater than one and factorial loads greater than 0.40. Three factors with eigenvalue greater than one were extracted. The oblique and varimax rotations were conducted in an attempt to improve their interpretation. In both methods of rotation, the same factorial structure was observed, without presenting a clear theoretical explanation for the dimensions obtained. For a better interpretation, the extraction of four factors was pre-defined. In addition, the oblique rotation method was used. This option is preferred when the objective of factorial analysis is to extract theoretical meanings or constructs and presumes a relationship between the factors¹⁵.

RESULTS

Characteristics of respondents

Of the 900 instruments completed by the respondents, 66 were answered incorrectly and excluded from the study. The 834 valid instruments equaled approximately 60% of the population of 45 participating clinics who were undergoing physical therapy treatment covered by private health insurance during the data collection period.

The respondents who took part in the study consisted of patients with a mean age of 46.7 years (SD= 15.8, range 18 - 92), of which 64.4% were female subjects. On average, the patients underwent 15 physical therapy sessions at the clinic where their data were collected. The most prevalent physical therapy specialties were: orthopedics/traumatology, rheumatology and neurology (57.4%, 22.5% and 7.0% respectively).

Reliability

The reliability, calculated through Cronbach's alpha coefficient, for the measures obtained with the instrument, was 0.94 for the final instrument.

Validity

Content validity

The content validity of the instrument was considered satisfactory because, during the creation of its items, the main aspects which represent the construct of patient satisfaction were taken into account. These items represent the domains cited in the literature on satisfaction.

In general, the items which composed the questionnaire were part of instruments of important studies published on satisfaction, whose psychometric properties, such as reliability and validity, were previously tested. In addition, qualitative expert analysis of the items, the application of the pre-test, the cultural adaptation, and later refinement of the instrument for the studied population assured appropriate format and vocabulary for the purpose of the measurement.

Simultaneous validity

To assess simultaneous validity, 3 items considered the best satisfaction indexes were removed to function as criterion measures. The 20 remaining items were used to form a summary score with the rest of the instrument. The correlations obtained for the criterion variables and summary score were the following: r= 0.71 (p 0.01) for question 21, "Overall satisfaction with your experience with physical therapy"; r= 0.60 (p 0.01) for question 22, "Would you return to this clinic if, in the future, you need physical therapy treatment?"; and r= 0.60 (p 0.01) for question 23, "Would you recommend this clinic to your family and friends?".

Construct validity

The inter-item correlation matrix demonstrated that all other items correlated positively and significantly with overall satisfaction measures (p< 0.01). These measures present, in general, greater correlations with issues related to patient-therapist interaction and smaller correlations with issues regarding convenience, such as clinic location and parking. The inter-item correlation matrix also showed that higher correlations occurred among items pertaining to the same dimension of patient satisfaction with physical therapy.

The investigation of sampling adequacy (KMO 0.95) indicated the relevance of the factorial model. The four pre-defined factors explained 64.49% of the total variance. Only one retained factor obtained a smaller eigenvalue, although very close to 1. However, this fourth factor was still considered, as it generated a dimension that is widely discussed in the literature on patient satisfaction. All items of the instrument presented factorial loads greater than 0.65, indicating a highly significant degree of correlation between the variable and the considerable factor.

The extracted components, the accounted for variation for each factor and the accumulated variation, together with Cronbach's alpha coefficient values for each component, are presented in Table 1.

It is possible to observe, in Table 2, that, after oblique rotation, the first component retained 8 items related to interpersonal aspects of the patient-therapist interaction. The second component retained 2 items relative to physical environment. The third component retained 6 items regarding patient access to care and regarding reception and support staff assistance. The fourth component retained 2 items related to convenience such as parking and location of the facility. Table 2 also indicates the mean and standard deviation of each index.

DISCUSSION AND CONCLUSIONS

The results obtained in the investigation of reliability and validity of the original Brazilian instrument to measure patient satisfaction with physical therapy indicated satisfactory psychometric properties for its use in clinical practice.

The reliability determined through Cronbach's alpha coefficient (α = 0.94) for the 23-item instrument exceeded the values proposed as criterion for exploratory studies^15,22. Among the subscales interpreted in the investigation of instrument validity, only the convenience subscale did not present satisfactory reliability. It is possible that this fact is attributed to the low number of items retained in this factor^19,23.

As in previous studies, our findings also support the evidence for greater correlations between items of the same domain^2,8,12. These results are consistent with the construct validity of the instrument^2,19.

Our results suggest better reliability as well as simultaneous and construct validities for the subscale which united the items that are directly related to patient-therapist interaction. These findings are in agreement with previous studies that also indicated this dimension as the most correlated with physical therapy patient satisfaction^2,8,12. Blanchard et al.²⁴ stated that these aspects form the main component of patient satisfaction.

The issues which are not directly related to interpersonal aspects of physical therapy (i.e., access, physical environment, convenience) showed less internal consistency. Some authors have already suggested that convenience-related aspects, such as parking and clinic location, do not measure the construct of satisfaction to the same extent as the other questions^2,8,12,25,26.

Roush and Sonstroem¹¹, on the other hand, stated that patient satisfaction is strongly influenced by factors such as location and costs. In addition, McIver²⁷ proposed that, in the outpatient context, accessibility, waiting time and the attitude of the support staff are critical components for patient satisfaction.

Similar factorial structure was identified in a previous study within the same cultural context which used the Portuguese-language version of the instrument developed by Goldstein et al.² and culturally adapted and validated for the Brazilian population¹².

The dimension most associated to the perception of quality of care according to patient satisfaction presented as main indexes the aspects related to the therapist's friendliness and communication skills. These items showed high factorial loads for this factor. Our results suggest that actions such as showing confidence during care, clarifying patients' questions, providing them opportunities to express their opinion, politeness and respect to their privacy can be the main sources of patient satisfaction and/or discontent.

These results not only show that patient-therapist interaction is strongly correlated with satisfaction, but also indicate that the communication between the professional and the patient may be the main connection between the various aspects of the process of physical therapy. Keith²⁸ had already commented that the perception of quality of care in rehabilitation is influenced by displays of concern and affection, by aspects of interpersonal relationship and by the evidence of technical competence, and that the nature and extension of the communication which takes place during consultation covers all these factors and is also part of satisfaction.

In our opinion, the data obtained in this study are applicable to privately insured adult patients undergoing outpatient physical therapy. The adequacy of the instrument for uninsured patients is unknown because aspects of cost were not addressed.

Our instrument was created to assess satisfaction with care and is not appropriate to measure patient satisfaction with the results obtained. According to Beattie et al.⁸, the measure of patient satisfaction with the result of physical therapy should include other relevant measures, such as health condition, functional capacity and quality of life.

Finally, the results accomplished in this study provide an instrument to measure patient satisfaction with physical therapy care. Moreover, by identifying the satisfaction indexes which patients consider when forming their opinion on the quality of care, it is possible to contribute not only to the administrative procedure, but also to the entire planning process needed to improve the quality of physical therapy services.

REFERENCES

• 1. Franco SC, Campos GWS. Avaliação da qualidade de atendimento ambulatorial em pediatria em um hospital universitário. Cad Saúde Pública. 1998;14:61-70.
• 2. Goldstein MS, Elliot SD, Guccione AA. The development of an instrument to measure satisfaction with physical therapy. Phys Ther. 2000;80:853-63.
• 3. Linder-Pelz SU. Toward a theory of patient satisfaction. Soc Sci Med. 1982;16:577-82.
• 4. Donabedian A. La dimensión internacional da la evaluación y garantía de calidad. Salud Publ Mex. 1990;32:113-7.
• 5. Bacthelor C, Owens DJ, Bloor M. Patient satisfaction studies: methodology, management and consumer evaluation. Int J Health Care Qual Assur. 1994;7:22-30.
• 6. Donabedian A. The quality of care: how can it be assessed? J Am Med Assoc. 1988;260:1743-8.
• 7. Mendonça KMPP, Guerra RO, Diógenes TPM, Lima TMFS. Influência das características sócio-demográficas na satisfação do paciente com o tratamento fisioterapêutico. Fisioter Mov. 2006;19:83-9.
• 8. Beattie PF, Pinto MB, Nelson MK, Nelson R. Patient satisfaction with outpatient physical therapy: Instrument validation. Phys Ther. 2002;82:557-65.
• 9. Monnin D, Perneger TV. Scale to measure patient satisfaction with physical therapy. Phys Ther. 2002;82:682-91.
• 10. Chistopher M, Oermann MD, Swank PR, Sockrider MM. Validation of an instrument measuring patient satisfaction with chest physiotherapy techniques in cistic fibrosis. Chest. 2000;118:92-7.
• 11. Roush SE, Sonstroen RJ. Development of the physical therapy outpatient satisfaction survey (PTOPS). Phys Ther. 1999; 79:159-70.
• 12. Mendonça KMPP. Satisfação do paciente com a fisioterapia: tradução, adaptação cultural e validação de um instrumento de medida [dissertação]. Natal (RN): UFRN; 2004.
• 13. Pasquali L. Instrumentos psicológicos: manual prático de elaboração. 1Ş ed. Brasília: LabPAM/IBAPP; 1999.
• 14. Ciconelli RM. Medidas de avaliação de qualidade de vida. Rev Bras Reumatol. 2003;43:9-13.
• 15. Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate data analysis. New Jersey: Prentice Hall; 1998.
• 16. Jackson JL, Chamberlin J, Kroenke K. Predictors of patient satisfaction. Soc Sci Med. 2001;52:609-20.
• 17. Cohen C. Age and health status in a patient satisfaction survey. Soc Sci Med. 1996;42:1085-93.
• 18. Dufrene RL. An evaluation of a patient satisfaction survey: validity and reliability. Eval Prog Plann. 2000;23:293-300.
• 19. Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. 1Ş ed. Brasil: Vozes; 2003.
• 20. Bunchaft G, Cavas CST. Sob medida: um guia sobre a elaboração de medidas do comportamento e suas aplicações. Brasil: Vetor; 2002.
• 21. Cobos JP, Valero S, Haro G, Fidel G, Escuder G, Trujols J, et al. Development and psychometric properties of the Verona Service Satisfaction Scale for methadone-treated opioid-dependent patients (VSSS-mT). Drug Alcohol Depend. 2002;68:209-14.
• 22. Malhotra NK. Pesquisa de marketing: uma orientação aplicada. 3Ş ed. Montigelli Jr N, Farias AA, tradutores. Porto Alegre: Bookman; 2001. 719p.
• 23. Hasselmann MH, Reichenheim ME. Adaptação transcultural da versão em português da Conflict Tactics Scales Form R (CTS-1), usada para aferir violência no casal: equivalências semânticas e de mensuração. Cad Saúde Pública. 2003;19: 1083-93.
• 24. Blanchard CG, Labrecque MS, Ruckdeschel JC, Blanchard EB. Physician behaviors, patient perceptions, and patient characteristics as predictors of satisfaction of hospitalized adult cancer patients. Cancer. 1990;65:189-92.
• 25. Pascoe GC, Attkisson CC. Comparison of indirect and direct approaches to measuring patient satisfaction. Eval Prog Plann. 1983;6:359-71.
• 26. Etter JF, Perneger TV. Validating a satisfaction questionnaire using multiple approaches: a case study. Soc Sci Med. 1997;45: 879-85.
• 27. McIver S. Obtaining the views of outpatients. London: King's Fund; 1991.
• 28. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil. 1998;79:1122-8.

Appendix 1

Publication Dates

• Publication in this collection
  11 Dec 2008
• Date of issue
  Oct 2007

History

• Reviewed
  10 Apr 2007
• Received
  06 Nov 2006
• Accepted
  25 June 2007