Content validity of the scale Perceptions on Interpersonal Racial Discrimination in Brazilian Healthcare Facilities (Driss)¹

Abstract

Aiming to elaborate and assess the content validity of the new scale Perceptions on Interpersonal Racial Discrimination in Brazilian Healthcare Facilities (Driss), we conducted a methodological study based on the Classical Test Theory. Initially, we had a pool of 49 items based on a comprehensive literature review, previous qualitative study, and recommendations from a research group on ethnic-racial inequalities. To assess content validity, an experts committee was formed. Qualitative and quantitative criteria were used to ensure methodological rigor. After the evaluation of the Driss version 1 by the experts committee, 28 items were excluded; 10 others were fragmented and/or modified. Thus, Driss version 2 was developed and subjected to pre-test, which showed a need to include introductory questions to make it better understood by the target population regarding the objectives of the study. Subsequently, we elaborated Driss version 3, whose content aims to value perceptions and feelings experienced by individuals who suffered racial discrimination at healthcare facilities. The content validity of version 3 was assessed by the Content Validity Ratio, being considered satisfactory and ready for later construct validity valuation.

Keywords:
Questionnaire; Psychometrics; Racial Discrimination; Racism

Resumo

Com o objetivo de elaborar e validar o conteúdo do instrumento de medida das Percepções sobre Discriminação Racial Interpessoal nos Serviços de Saúde Brasileiros (Driss), foi conduzido um estudo do tipo metodológico com base na Teoria Clássica dos Testes. O pool de itens inicial, com 49 itens, foi elaborado a partir dos resultados de uma revisão abrangente de literatura, um estudo qualitativo prévio e recomendações de um grupo de pesquisa sobre desigualdades étnico-raciais. Para a validação de conteúdo, um comitê de especialistas foi formado. Critérios qualitativos e quantitativos foram empregados para garantir rigor metodológico. Após a avaliação da versão 1 do Driss pelo comitê de especialistas, houve exclusão de 28 itens, além do desmembramento e/ou modificação de outros dez. A partir disso, elaborou-se a versão 2, submetida a pré-teste, que mostrou a necessidade de se incluírem questões introdutórias para melhor entendimento por parte da população-alvo quanto aos objetivos do estudo. Posteriormente, elaborou-se a versão 3, com conteúdo que busca valorizar as percepções e sentimentos experimentados pelos indivíduos que sofreram discriminação racial nos serviços de saúde. A validade de conteúdo da versão 3 foi verificada, por meio da Razão de Validade de Conteúdo, sendo considerada satisfatória e pronta para a verificação da validade de constructo.

Palavras-chave:
Questionário; Psicometria; Discriminação Racial; Racismo

Introduction

Racial discrimination refers to the differentiated treatment provided to people based on their race/color, which may manifest through actions of favor or prejudice towards one group over the other, privileging the group to which the perpetrator belongs (Priest; Williams, 2018). Racial discrimination can occur at the institutional and interpersonal levels: institutional racism refers to institutional policies and practices that privilege one racial group over another (Werneck, 2016); interpersonal racism - which is the object of this study - refers to the different treatment based on race/color manifested through actions of favor or prejudice towards an individual in relation to another individual (Jones, 2000).

In recent decades, part of the international scientific community has been investigating racial discrimination and its repercussions on the health of individuals. A systematic review showed the negative effects of racism on general, physical, and mental health conditions, such as psychological stress, anxiety, high blood pressure, and obesity (Paradies et al., 2015).

Despite this evidence, the idea that health inequalities stem from a legacy of socioeconomic disadvantage affecting the black population persists (Batista, 2019; Williams; Lawrence; Davis, 2019). In this paper, black population (negros, in Brazilian Portuguese) is considered the group of people self-classified as black and pardo (light-skin) (Werneck, 2016), with the black population (negros) amounting to 54.8% of the Brazilian population (População…, 2017). However, within the Brazilian context, data reveal that the black population faces disadvantageous health outcomes when compared to the white population (Retrato..., 2017); for example, black women receive less analgesia during childbirth, when necessary (Leal et al., 2017), and also initiate prenatal care later, as well as need to search much more for hospitals at the time of delivery (Viellas et al., 2014). As for black men, mortality rates are higher than that of white men regarding practically all causes of death (Souza; Araújo; Nery, 2015).

That said, this study assumes that racial discrimination is one of the determinants of the inequalities demonstrated by health indicators in Brazil when comparing black and white populations, corroborating the findings of other studies (Chehuen Neto et al., 2015). In this sense, part of the racial discrimination suffered by patients seems to originate from the interaction with health professionals during the consultations (Paradies; Truong; Priest, 2014). For this reason, we decided to give voice to the perceptions of those patients who may have been discriminated against due to their race or skin color while seeking care in Brazilian healthcare facilities.

We sought an instrument in both national and international literature to measure interpersonal racial discrimination suffered by patients and perpetrated by health professionals. The instruments available up to that moment were mostly from the United States and had no specific focus on healthcare facilities (Bastos et al., 2010; Kressin; Raymond; Manze, 2008; Paradies; Truong; Priest, 2014). As for the few instruments developed or adapted to the Brazilian reality, none of them evaluated the interpersonal racial discrimination perpetrated by health professionals (Bastos et al., 2012; Fattore et al., 2016; Santos et al., 2006). Given this scenario, as it reflects a gap in knowledge in the field of public health, we chose to elaborate and validate the instrument for measuring Perceptions on Interpersonal Racial Discrimination in Brazilian Healthcare Facilities (Driss) in this study.

Methodology

This is a methodological study, aimed at validating Driss’s content based on the methodological tools of the Classic Test Theory.

The elaboration of Driss’s items

The following steps were followed in the elaboration of Driss’s items: theoretical framework, development of the item pool - Driss v.1.0, and choice of response format (DeVellis, 2017; Furr, 2015).

Using DeVellis’s (2017) theoretical perspective, three pillars were used to establish Driss’ theoretical framework: a qualitative study, conducted with in-depth interviews; a comprehensive literature review, evaluating the existing scales for measuring discrimination; and the recommendations of a research group on ethnic-racial inequalities in health (DeVellis, 2017).

The qualitative study used was the dissertation entitled Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda (Santa Rosa, 2013). The study was conducted with 20 low-income black women living in a majoritarily black community on the outskirts of the capital of São Paulo, Brazil, in 2013. The testimonies collected demonstrated the women’s perceptions about interpersonal racial discrimination in the healthcare facilities context. Based on their personal and witnessed experiences, they reported that racial discrimination occurred mostly during the interaction with health professionals - therefore, we decided to include this work, as it comprises opinions from patients; in other words, the target population of the study (DeVellis, 2017).

The comprehensive literature review was conducted through the assessment of systematic and non-systematic reviews on instruments for measuring discrimination found up to 2014 (Bastos et al., 2010; Kressin; Raymond; Manze, 2008; Paradies; Truong; Priest, 2014) which indicated the existing discrimination scales. Mostly developed in the United States, the scales were individually evaluated, and the items matching the objectives of this study were incorporated into Driss’ first version (Bastos et al., 2010; Kressin; Raymond; Manze, 2008; Paradies; Truong; Priest, 2014).

Still regarding Driss’s theoretical framework, the suggestions of a research group on ethnic-racial inequalities in health, registered with the National Council for Scientific and Technological Development (CNPq), were taken into consideration. This group is linked to a postgraduate program in public health at a public higher education institution in the state of Bahia, Brazil. The initial item pool was introduced at a specific meeting in 2015, and teachers, masters, doctors, and postgraduate students made important contributions to the reformulation and elaboration of the instrument items.

Choice of response format

To choose the response format, it was necessary to decide whether the instrument would measure the presence/absence or frequency of the phenomenon under investigation (DeVellis, 2017). By consensus of the research group on ethnic-racial inequalities in health, we started from the understanding that racial discrimination really occurs within the context of these facilities, which means that the phenomenon exists - therefore, discarding the need for measuring its presence/absence. Thus, we set for measuring the frequency of experiences, feelings, reactions, and attitudes of those involved. Possible responses were “rarely,” “sometimes,” “many times,” “regularly,” and “always.” At the end of the three steps, Driss’s first version was prepared and then submitted to content validation process.

Content validity

Content validation is the process of verifying the suitability of a set of items for the representation of a given phenomenon, that is, within a universe of possible items, only items that reflect the investigated construct should remain in the final instrument (DeVellis, 2017). Considering that the phenomenon object of this study was the interpersonal racial discrimination practiced within the healthcare facilities context, we sought a set of items that reflected it as much as possible, which was carried out until the design of Driss v.1.0 (^{Appendix 1}).

Experts committee

An experts committee composed of a methodologist, three specialists in ethnic-racial relations, a linguist, and three members of the target population was formed to assess the content validity of Driss v.1.0, based on what is proposed by Coluci, Alexandre, and Milani (2015).

The experts were invited to join the Driss evaluation committee by email between February and March 2017. The answer was expected within 20 days; when that time was reached or when the invitation was declined, another specialist with the same experience within the same area of knowledge was sought after.

When the expert agreed to participate in the study and the committee, an email containing the link to Driss v.1.0’s evaluation form, built at the digital platform Research Electronic Data Capture (REDCap) (Harris et al., 2009), was sent to the new member. Also, the entire research project and proof of approval from the Research Ethics Committee (CEP) were sent.

Qualitative and quantitative criteria were considered (Coluci; Alexandre; Milani, 2015) for the verification Driss’s content validity. As qualitative criteria, the relevance, clarity, and adequacy of the sample were assessed (Wilson; Pan; Schumsky, 2012) through the following options: (1) relevance (irrelevant; relevant, but demands major revision; relevant, but demands minor revision; relevant); (2) clarity (unclear; relatively clear; very clear); (3) dimension (reaction; experience; feeling; other); and (4) maintenance (kept unchanged; kept modified; excluded). This analysis was based on the evaluation of the committee’s written contributions. As for the question about the maintenance of the item, if the specialist selected the option “kept modified,” a text box would open to include suggestions for modification; if the chosen option were “excluded,” a text box would open asking for the justification. Besides, for each topic, there was a text box at the end for suggestions related to the item. These text boxes provided the specialists with space to present both their criticisms and suggestions regarding each item.

For the quantitative analysis of content validity, the question regarding the item’s maintenance (kept unchanged; kept modified; excluded) was used as a parameter. When the specialist chose “kept unchanged” or “kept modified,” the item was counted as a vote to keep (code 1); when their option was to “exclude,” the item was counted as a vote to exclude (code 0). Such data were analyzed using the Content Validity Ratio (CVR) - also known as Lawshe’s Content Validity Ratio. CVR is a linear statistic of the proportion of specialists who agree with the maintenance of the item: the cut-off point used was 0.75, that is, all items with a CVR greater than or equal to 0.75 were maintained and all items with a CVR lower than this value were excluded (Wilson; Pan; Schumsky, 2012).

After applying the CVR, an agreement test was carried out between the specialists, using the Kappa adjusted for prevalence (Pabak) (Byrt; Bishop; Carlin, 1993). Pabak values considered were: <0, no agreement; 0 to 0.20, poor agreement; 0.21 to 0.40, slight agreement; between 0.41 to 0.60, reasonable agreement; 0.61 to 0.80, good agreement; 0.81 to 0.92, very good agreement; and, finally, 0.93 to 1.00, excellent agreement (Byrt; Bishop; Carlin, 1993).

The members of the target population were people of legal age, literate, of sound mind, and who wished to participate in the research. Most of them belonged to research groups and educational activism groups present in social networks such as WhatsApp and Facebook. Thus, these same people were encouraged to disseminate the research through their respective networks.

Pre-test

Initially, Driss v.1.0’s pre-test was performed by filling out a printed form. Subsequently, it was self-completed directly on REDCap, accessed via a link sent by email, WhatsApp, or Facebook. We observed if the questions were well understood by the respondents and if the REDCap form worked properly; an effort was also in place to verify the need to include more items. Pre-test respondents were people from the research groups in which the authors of this work were inserted, as well as members of the target population. The estimated time for completion was 15 minutes.

Ethical procedures

All ethical procedures of Resolution No. 466/2012 of the Conselho Nacional de Saúde (Brasil, 2013) were respected during the entirety of this study, approved by the CEP of the Escola de Enfermagem da Universidade de São Paulo (CAAE No. 55713316.1.0000.5392).

Results

To validate Driss’s content, 18 people were invited to participate in the committee of experts. Among the professionals invited, there were two methodologists, three Portuguese-speaking professionals, seven specialists in ethnic-racial relations, and six members of the target population. As noted in Chart 1, seven people did not respond, one refused to participate, and two members of the target population did not understand the instructions to complete the form. In total, eight people agreed to participate, of which one was a methodologist, one was a Portuguese-speaking professional (under a paid contract), three were specialists in ethnic-racial relations, and three were members of the target population.

Chart 1
Summary of participation in the expert committee to validate the instrument for measuring Perceptions on Interpersonal Racial Discrimination in Brazilian Healthcare Facilities, according to the practice area and reason for participating, or not, São Paulo, 2017

Practice Area	Participant	Reason for not participating	Participation
Psychometrist/Methodologist	2	did not answer (1)	accepted (1)
Portuguese-speaking professional	3	did not answer (2)	accepted (1) (paid)
Ethnic-racial relations specialist	7	did not answer (3)	accepted (3)
		refused (1)
Target population representative	6	did not answer (1)	accepted (3)
		had difficulties while trying to understand the instructions (2)
Total	18	10	8

The methodologist was a Ph.D. and postdoc in psychometry, with extensive experience and numerous publications in the area. The Portuguese-speaking professional had previous experience in analyzing measurement instruments and correcting other types of academic work. The three specialists in ethnic-racial relations were Ph.D. professors from public universities in the Brazilian Northeast and Southeast regions. Regarding the members of the target population, two were lawyers and one was a nurse, all leaders of social movements (black and feminist) and indicated to fully understand what was required for the evaluation of the instrument, that is, both the written instructions and the social content of the work.

Driss v.1.0

Driss’s v.1.0 (^{Appendix 1}), precisely the version sent for analysis by the expert committee, was comprised of 49 items, in addition to the introductory text. Based on the experts’ contribution, both the introductory text and seven other items (2, 4, 14, 18, 19, 25, and 26) have been modified. Items 18 and 25 seemed to deal with more than one topic and thus were separated into different items: item 18 gave rise to four other topics and 25 to three others.

Items 5 to 8, 11, 13, 20 to 24, 27 to 44, and 48 were excluded. Although items 39 and 40 had a CVR score of 0.75, they were excluded for being closely related, and only understandable, in the presence of other items with similar content. Therefore, of the 49 items originally prepared, 19 remained; with the further splitting of items 18 and 25, 24 remained, which led to the development of Driss v.2.0. The average CVR of the remaining items was 0.74. The excluded items addressed subjects perceived as repetitive, confusing, or which included manifestations uncommon among Brazilians (artistic or literary manifestations as a coping strategy after perceiving themselves racially discriminated, such as drawing, singing, painting, and writing).

Then the agreement between the experts was verified with a Pabak score of 0.30 - therefore, it was considered a slight agreement. No item has been modified or excluded after this result, thus producing the second version of the instrument.

Driss v.2.0

Comprised of 24 items, this version (^{Appendix 2}) was submitted to members of the target population for pre-test between August and September 2017. Three interviews were conducted with members of the target population in August, all of them recorded on paper. The interviews allowed us to understand the need to include more introductory questions that would encourage people to focus on the research objectives, specifically on healthcare facilities. Thus, examples of types of healthcare facilities and nine examples of unfair treatment were inserted among the introductory questions, with the possibility of including some other situation in writing (^{Appendix 3}).

It was also observed that the respondents found item 24 especially difficult to understand. This item questioned whether the respondent had witnessed situations of privilege to third parties according to race/color: “I have witnessed an unknown person being treated with privilege by health care providers due to their race/skin color.” Thus, we decided to include the term “THE PERSON’S,” in capital letters, to facilitate the respondent’s understanding (^{Appendix 3}).

The three forms filled out on paper were typed into REDCap in September 2017 to test the platform. Thus, assistance was requested from classmates, the co-authors of this work, as well as members of the target population, to answer the instrument directly on REDCap, through the link provided. Small issues were identified while moving from one item into the next, which were solved. The pre-test was accessed 40 times in total, with 21 people fully answering the instrument. The pre-test demonstrated that answering the Driss online via REDCap was a feasible effort and that it was ready to be validated by the target population. At that time, the Driss v.3.0 (^{Appendix 4}) was elaborated and then made available for the target population to answer.

Driss v.3.0

Driss v.3.0 (^{Appendix 4}) was developed inside REDCap and used to analyze the construct’s validity (not described in this article). In summary, this version contains introductory questions seeking to identify in which healthcare facility the perception of the racially discriminatory act took place, a summarized description of the situation, which professional perpetrated it, their sex, race/color, and when it happened. Then there are four groups of items containing 24 items in total with the goal to assess perceptions, feelings, and attitudes towards the self-reported racially discriminatory situation in the healthcare facility context. With the completion of Driss 3.0, we could observe that the verification of the content validity was completed, having reached a satisfactory psychometric validity.

Discussion

This methodological study presented the content validation process of a scale to measure perceptions on interpersonal racial discrimination in healthcare facilities, called Driss. The first version, containing 49 items, was based on a literature review on the scales for measuring discrimination, the opinion of a research group on ethnic-racial inequalities in health, and from a qualitative study (Santa Rosa, 2013), as suggested by recent literature (DeVellis, 2017). For the initial version evaluation, a committee of experts was created, containing professionals considered relevant to the studied topic, as well as members of the target population, totaling eight people. After evaluated by the expert committee, the number of items was reduced to 24, making up Driss’s second version. In turn, the second version was pre-tested, giving rise to the third version, with 24 items split into four groups of questions.

Regarding the excluded items, three main groups were observed: the first concerned artistic or literary manifestations, such as drawing, singing, painting, and writing, as coping strategies after perceiving racially discriminated. The experts argued that it did not seem to be a matter of common practice by Brazilians in face of stressful situations. This argument made sense, given that the items originally belonged to a scale produced to assess discrimination suffered by Australian Natives (Paradies; Cunningham, 2008), having been included as a test in Driss v.1.0, as suggested by literature (DeVellis, 2017; Furr, 2015).

The second group of items, which included, for example, “complaining in a discrete tone of voice,” “making a scene,” and the fear of “being humiliated, mistreated, refused assistance, receiving wrong medication,” was assessed as confusing or repetitive. These were timely arguments, and several of these exclusions had been previously anticipated by the authors. In fact, the initial item pool should even include similar items, with different wording, to test which version would be the most appropriate - according to the literature used as a basis (DeVellis, 2017; Furr, 2015), in advance, the author is aware that many items will be excluded from the final version of a scale in development.

The third group of items, which referred to the healthcare professionals’ race/skin color, was also excluded. Such exclusion aroused curiosity, as it was contrary to studies that present data indicating that the race/skin color of the healthcare professionals can interfere in the perception of racial discrimination (Wren Serbin; Donnelly, 2016). Although there was some estrangement on the part of the researchers, the suggestion was accepted.

As for the items which remained, there was an appreciation of the feelings experienced, as well as reactions to the racially discriminatory act. Perhaps because this type of racial discrimination is often covert, there seems to be some difficulty in its identification and possible complaints, as racism is considered a crime in Brazil (Brasil, 1989). However, the patients can perceive racial discrimination targeted to them and be able to express their impressions about it; such racial discrimination can result in limited access or even occasional withdrawal from treatment.

Also, the Driss introductory questions facilitated the respondent’s understanding of the study’s objectives, reinforcing that they must complete the instrument considering situations of discrimination motivated exclusively by the patient’s race/skin color, and in the specific environment of healthcare facilities.

As a limitation of this study, we must also consider that, in Brazil, a large part of the population has low education and is socio-economically disadvantaged. It is precisely in this group that the black population also predominates, and where digital exclusion is more frequent (CGI, 2018). Therefore, before using REDCap or any digital medium for data collection, these tools’ potential and limitations must be considered.

Final remarks

Concluding, Driss’s development followed the most rigorous recommendations present in the specific literature so far. All steps for checking the content validity have been properly carried out. The initial item pool was submitted to a competent experts committee, in which quantitative and qualitative criteria were considered for the response analysis. The inclusion of introductory questions facilitated a more objective filling of the form. The instrument includes items that deal with reactions, attitudes, and feelings towards the perpetrator of the racially discriminatory act, as well as perceptions related to having witnessed other people being racially discriminated against in the healthcare facilities contexts, considering the theoretical framework - Driss is an attempt to highlight and value the perceptions and feelings experienced. It seems easy to fill out and can be applied to the entire adult population who are cognitively able to respond the form, regardless of their race/skin color. It is an instrument with the potential to be used in surveys or epidemiological studies focusing on healthcare facilities.

Furthermore, Driss may be considered the only instrument available in Portuguese, so far, capable of measuring the perception of racial discrimination specifically practiced by healthcare professionals, since the respondents are the people using those facilities. It can be proven useful for studies carried out in Brazil, where racial discrimination usually takes a covert form, corroborating the myth of “racial democracy” (Fernandes, 2007) and making racial discrimination difficult to measure and combat.

However, Driss’s version resulting from the construct validation process is the best for use with the target population, as it is the version that gathers more validity evidence: this would be Driss v.4.0, which will be presented in due course in a future publication.

Referências

• BASTOS, J. L. et al. Racial discrimination and health: a systematic review of scales with a focus on their psychometric properties. Social Science & Medicine, Oxford, v. 70, n. 7, p. 1091-1099, 2010.
• BASTOS, J. L. et al. Explicit discrimination and health: development and psychometric properties of an assessment instrument. Revista de Saúde Pública, São Paulo, v. 46, n. 2, p. 269-278, 2012.
• BATISTA, L. E. Projeto: análise de políticas de saúde no Brasil. [Entrevista concedida a] Observatório de Análise Política em Saúde. Análises de Políticas de Saúde, Salvador, 22 fev. 2019. Disponível em: <Disponível em: https://bit.ly/3nVf2Gq >. Acesso em: 4 jan. 2021.
  » https://bit.ly/3nVf2Gq
• BRASIL. Lei nº 7.716, de 5 de janeiro de 1989. Define os crimes resultantes de preconceito de raça ou de cor. Diário Oficial da União, Brasília, 6 jan. 1989. Disponível em: <Disponível em: https://bit.ly/39MEzwa >. Acesso em: 4 jan. 2021.
  » https://bit.ly/39MEzwa
• BRASIL. Ministério da Saúde. Conselho Nacional de Saúde. Resolução nº 466, de 12 de dezembro de 2012. Aprova diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos. Diário Oficial da União, Brasília, DF, 13 jun. 2013. Seção 1, p. 59.
• BYRT, T.; BISHOP, J.; CARLIN, J. B. Bias, prevalence and kappa. Journal of Clinical Epidemiology, Oxford, v. 46, n. 5, p. 423-429, 1993.
• CGI - COMITÊ GESTOR DA INTERNER NO BRASIL. TIC domicílios: pesquisa sobre o uso das tecnologias de informação e comunicação nos domicílios brasileiros: 2018. São Paulo, 2018. Disponível em: <Disponível em: https://bit.ly/3paoWoT >. Acesso em: 20 jan. 2021.
  » https://bit.ly/3paoWoT
• CHEHUEN NETO, J. A. et al. Política Nacional de Saúde Integral da População Negra: implementação, conhecimento e aspectos socioeconômicos sob a perspectiva desse segmento populacional. Ciência & Saúde Coletiva, Rio de Janeiro, v. 20, n. 6, p. 1909-1916, 2015.
• COLUCI, M. Z. O.; ALEXANDRE, N. M. C.; MILANI, D. Construção de instrumentos de medida na área da saúde. Ciência & Saúde Coletiva, Rio de Janeiro, v. 20, n. 3, p. 925-936, 2015.
• DEVELLIS, R. F. Scale development: theory and applications. 4. ed. Los Angeles: Sage, 2017.
• FATTORE, G. L. et al. Validade de constructo da escala Experiences of Discrimination em uma população brasileira. Cadernos de Saúde Pública, Rio de Janeiro, v. 32, n. 4, e00102415, 2016.
• FERNANDES, F. Aspectos da questão racial. In: FERNANDES, F. O negro no mundo dos brancos 2. ed. São Paulo: Global, 2007. p. 43.
• FURR, R. M. Scale construction and psychometrics for social and personality psychology Washington, DC: Sage, 2015.
• HARRIS, P. A. et al. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, San Diego, v. 42, n. 2, p. 377-381, 2009.
• JONES, C. P. Levels of racism: a theoretic framework and a gardener’s tale. American Journal of Public Health, Washington, DC, v. 90, n. 8, p. 1212-1215, 2000.
• KRESSIN, N. R.; RAYMOND, K. L.; MANZE, M. Perceptions of race/ethnicity-based discrimination: a review of measures and evaluation of their usefulness for the health care setting. Journal of Health Care for the Poor and Underserved, Thousand Oaks, v. 19, n. 3, p. 697-730, 2008.
• LEAL, M. C. et al. A cor da dor: iniquidades raciais na atenção pré-natal e ao parto no Brasil. Cadernos de Saúde Pública, Rio de Janeiro, v. 33, e00078816, 2017. Suplemento 1.
• PARADIES, Y.; CUNNINGHAM, J. Development and validation of the Measure of Indigenous Racism Experiences (MIRE). International Journal for Equity in Health, London, v. 7, n. 1, art. 9, 2008.
• PARADIES, Y.; TRUONG, M.; PRIEST, N. A systematic review of the extent and measurement of healthcare provider racism. Journal of General Internal Medicine, Philadelphia, v. 29, n. 2, p. 364-387, 2014.
• PARADIES, Y. et al. Racism as a determinant of health: a systematic review and meta-analysis. Plos One, San Francisco, v. 10, n. 9, e0138511, 2015.
• POPULAÇÃO chega a 205,5 milhões, com menos brancos e mais pardos e pretos. Agência IBGE Notícias, Brasília, DF, 24 nov. 2017. Disponível em: <Disponível em: https://bit.ly/3bRYhto >. Acesso em: 4 jan. 2021.
  » https://bit.ly/3bRYhto
• PRIEST, N.; WILLIAMS, D. R. Racial discrimination and racial disparities in health. In: MAJOR, B.; DOVIDIO, J. F.; LINK, B. G. (Ed.). The Oxford handbook of stigma, discrimination and health New York: Oxford University Press, 2018. p. 163-182.
• RETRATO das desigualdades de gênero e raça em 20 anos. Instituto de Pesquisa Econômica Aplicada, Brasília, DF, 6 mar. 2017. Disponível em: <Disponível em: https://bit.ly/39HjHXo >. Acesso em: 4 jan. 2021.
  » https://bit.ly/39HjHXo
• SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertação (Mestrado em Ciências) - Universidade de São Paulo, São Paulo, 2013.
• SANTOS, W. S. et al. Escala de racismo moderno: adaptação ao contexto brasileiro. Psicologia em Estudo, Maringá, v. 11, n. 3, p. 637-645, 2006.
• SOUZA, I. M.; ARAÚJO, E. M.; NERY, F. S. Mortalidade masculina segundo a raça/cor da pele em estados do Nordeste e Sudeste do Brasil, 2009-2011. Revista da Associação Brasileira de Pesquisadores(as) Negros(as), Uberlândia, v. 7, n. 16, p. 154-167, 2015.
• VIELLAS, E. F. et al. Assistência pré-natal no Brasil. Cadernos de Saúde Pública, Rio de Janeiro, v. 30, p. S85-S100, 2014. Suplemento 1.
• WERNECK, J. Racismo institucional e saúde da população negra. Saúde e Sociedade, São Paulo, v. 25, n. 3, p. 535-549, 2016.
• WILLIAMS, D. R.; LAWRENCE, J. A.; DAVIS, B. A. Racism and health: evidence and needed research. Annual Review of Public Health, Palo Alto, v. 40, n. 1, p. 105-125, 2019.
• WILSON, F. R.; PAN, W.; SCHUMSKY, D. A. Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, London, v. 45, n. 3, p. 197-210, 2012.
• WREN SERBIN, J.; DONNELLY, E. The impact of racism and midwifery’s lack of racial diversity: a literature review. Journal of Midwifery & Women’s Health, New York, v. 61, n. 6, p. 694-706, 2016.

• 1
  A project funded by the National Council for Scientific and Technological Development (CNPq).
• 2
  These appendixes contain only a simple translation, just for the purpose of showing the work’s content in English. The authors reinforce that this instrument’s English version was not submitted to a validation process according to methodological rigor criteria recommended in the literature.

Appendix 1

Driss v.1.0
Item Pool ²

When I am treated unfairly due to my race/skin color, I:	Rarely	Sometimes	Many times	Regularly	Always	Source
1. Ignore it						Experiences of discrimination KRIEGER, N. Racial and gender discrimination: risk factors for high blood pressure? Social Science & Medicine, Oxford, v. 30, n. 12, p. 1273-1281, 1990. doi: 10.1016/0277-9536(90)90307-E Measure of indigenous racism experiences PARADIES, Y.; CUNNINGHAM, J. Development and validation of the Measure of Indigenous Racism Experiences (MIRE). International Journal for Equity in Health, London, v. 7, n. 1, art. 9, 2008. doi: 10.1186/1475-9276-7-9
2. see it as a part my of life
3. talk to someone, like a family member, or friend, about what happened
4. forget about what happened
5. write about what happened
6. make drawings about what happened
7. sing about what happened
8. paint about what happened
9. say a prayer (or pray) for the situation.
10. keep it to myself
11. complain in a discrete tone of voice to people who are close to me						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
12. complain out loud
13. make a scene
14. report it to the authorities (e.g.: I talk to the police or file a lawsuit).
15. feel humiliated						Specialists from the Núcleo Interdisciplinar de Estudos sobre Desigualdades em Saúde
16. feel helpless
17. feel angry
18. develop some physical symptom, such as: headaches, stomachache, muscle tension, or heart palpitation						Measure of indigenous racism experiences PARADIES, Y.; CUNNINGHAM, J. Development and validation of the Measure of Indigenous Racism Experiences (MIRE). International Journal for Equity in Health, London, v. 7, n. 1, art. 9, 2008. doi: 10.1186/1475-9276-7-9
19. am afraid I'll have to wait longer than normal to be assisted.						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
20. am afraid of being humiliated
21. am afraid of being mistreated
22. am afraid of being denied assistance						Specialists from the Núcleo Interdisciplinar de Estudos sobre Desigualdades em Saúde
23. am afraid of receiving the wrong medication						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
24. am afraid of receiving the wrong treatment
To avoid being treated unfairly due to my race/skin color, I:
25. take extra care with my appearance (hygiene, hair removal, etc.)						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
26. refrain from complaining about the service, even if I am suffering or in pain
27. go to a different service location						Specialists from the Núcleo Interdisciplinar de Estudos sobre Desigualdades em Saúde
28. no longer seek service
In my opinion:
29. White men are the ones who receive the most unfair treatment in a healthcare facility						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
30. White women are the ones who receive the most unfair treatment in a healthcare facility
31. Asian men are the ones who receive the most unfair treatment in a healthcare facility
32. Asian women are the ones who receive the most unfair treatment in a healthcare facility
33. Brown men are the ones who receive the most unfair treatment in a healthcare facility
34. Brown women are the ones who receive the most unfair treatment in a healthcare facility
35. Black men are the ones who receive the most unfair treatment in a healthcare facility
36. Black women are the ones who receive the most unfair treatment in a healthcare facility
37. Indigenous men are the ones who receive the most unfair treatment in a healthcare facility
38. Indigenous women are the ones who receive the most unfair treatment in a healthcare facility
I usually feel unfairly treated due to my race/skin color when, among health professionals (doctors, nurses, attendants, etc.)
39. There are only white/caucasian people						Literature recommendation BEARD, K. V.; VOLCY, K. Increasing minority representation in nursing. American Journal of Nursing, New York, v. 113, n. 2, p. 11, 2013. doi: 10.1097/01.NAJ.0000426668.44751.f8
40. There are only black/brown people
41. There are only white/caucasian and Asian people
42. There are only white/caucasian, Asian, and mixed-race people
43. There are only white/caucasian, Asian, brown, and black people
44. There are people of all races and colors
45. Have I ever witnessed a family member, relative, or friend being treated unfairly in a healthcare facility due to their race/skin color?						Dissertation SANTA ROSA, P. L. F. Mulheres negras, o cuidado com a saúde e as barreiras na busca por assistência: estudo etnográfico em uma comunidade de baixa renda. 2013. Dissertation (master's in sciences) - Universidade de São Paulo, São Paulo, 2013.
46. Did I ever witness other people being treated unfairly in a healthcare facility due to their race/skin color?
47. I was/felt like I was treated with privilege in a healthcare facility due to my race/skin color						Literature recommendation PARADIES, Y.; TRUONG, M.; PRIEST, N. A systematic review of the extent and measurement of healthcare provider racism. Journal of General Internal Medicine, Philadelphia, v. 29, n. 2, p. 364-387, 2014.
48. I have already witnessed a family member, relative, or friend being treated with privilege in a healthcare facility due to their race/skin color
49. I have witnessed an unknown person being treated with privilege in a healthcare facility due to their race/skin color

Appendix 2

Driss v.2.0

Item	Options
Introductory text:	Rarely	Sometimes	Many times	Regularly	Always
In this study, we only included questions that seek to investigate the situations in which you were treated unfairly in a healthcare facility due to your race/skin color.
When I am treated unfairly in a healthcare facility due to my race/skin color, I:
1. ignore it
2. see it as an everyday event
3. talk to someone, like a family member, or friend, about what happened
4. try to forget about what happened
5. say a prayer (or pray) for the situation.
6. keep it to myself
7. feel humiliated
8. feel helpless
9. feel angry
10. have headache
11. have stomachache
12. feel my muscles get tense
13. have palpitations
14. am afraid I’ll have to wait longer than normal to be assisted
15. complain out loud
16. report it to the authorities (e.g.: I register the incident with the police or file a lawsuit)
To avoid being treated unfairly due to my race/skin color when I’m using healthcare facilities, I:
17. take extra care with my hygiene
18. take extra care with my hair removal/shaving
19. take extra care with my clothing
20. refrain from complaining about the service provided
Answer the next questions while thinking about situations that happened to other people:
21. Have I ever witnessed a family member, relative, or friend being treated unfairly in a healthcare facility due to their race/skin color?
22. Did I ever witness other people being treated unfairly in a healthcare facility due to their race/skin color?
Answer the next questions by thinking about situations in which you have observed privilege:
23. I was/felt like I was treated with privilege in a healthcare facility due to my race/skin color
24. I have witnessed an unknown person being treated with privilege in a healthcare facility due to their race/skin color.

Appendix 3

Driss v.3.0 (with introductory questions)

Introductory questions

1. Have you ever been treated unfairly, in relation to other people present, due to your race/skin color in any of these institutions? If so, respond thinking about the last time this happened.

• It didn’t happen

• Primary care center (UBS)

• Hospital

• Emergency Room

• Ambulatory

• Dentist office

• Maternity hospital/ward

• Emergency Mobile Care Service (Serviço de atendimento móvel de urgência, or SAMU)

• Hospital/Health Center Pharmacy

• Laboratory

• Regulation center

• Other. Where? _____________________

2. What unfair treatment did you receive due to your race/skin color? Remember that, according to your perception, the only difference between you and the other people present was your race/color.

• Some employee looked at you with contempt

• You were moved back in some queue

• You received less potent medication

• You received lower quality medication

• You were verbally abused

• You have undergone a medical or nursing procedure more painful than it should be

• The health professional did not examine you properly

• The professional didn’t seem to want to touch you

• You were physically abused by the health professional

• Other. Please, briefly describe what happened: ____________________________________________________________________

3. How long ago did this happen?

• Last year

• More than 1, but less than 5 years ago

• More than 5 years ago

4. Who was the professional who treated you unfairly due to your race/skin color?

• Concierge

• Receptionist

• Nurse

• Nursing technician

• Nursing assistant

• Doctor

• Psychologist

• Dentist

• Nutritionist

• Social worker

• Community health worker

• Other. Who? ________________________________________________

5. What was the gender of the professional who treated you unfairly?

• Male

• Female

• Other. What was the professional’ gender?

6. What was the race/skin color of the professional who treated you unfairly?

• White/Caucasian

• Black/African descendent

• Brown (pardo)/light-skin

• Asian

• Indigenous

Appendix 4

Driss v.3.0

Item	Options
Introductory text:	Rarely	Sometimes	Many times	Regularly	Always
In this study, we only included questions that seek to investigate the situations in which you were treated unfairly in a healthcare facility due to your race/skin color.
When I am treated unfairly in a healthcare facility due to my race/skin color, I:
1. ignore it
2. see it as an everyday event
3. talk to someone, like a family member, or friend, about what happened
4. try to forget about what happened
5. say a prayer (or pray) for the situation.
6. keep it to myself
7. feel humiliated.
8. feel helpless.
9. feel angry
10. have headaches;
11. have stomach aches;
12. my muscles get tense;
13. have palpitations
14. am afraid I'll have to wait longer than normal to be assisted.
15. complain out loud
16. report it to the authorities (e.g.: I register the incident with the police or file a lawsuit).
To avoid being treated unfairly due to my race/skin color when I'm using health services, I:
17. take extra care with my hygiene.
18. take extra care with my hair removal/shaving.
19. take extra care with my clothing.
20. refrain from complaining about the service provided
Answer the next questions while thinking about situations that happened to other people:
21. Have I ever witnessed a family member, relative, or friend being treated unfairly in a healthcare facility due to their race/skin color?
22. Did I ever witness other people being treated unfairly in a healthcare facility due to their race/skin color?
Answer the next questions by thinking about situations in which you have observed privilege:
23. I was/felt like I was treated with privilege in a healthcare facility due to my race/skin color
24. I have witnessed an unknown person being treated with privilege in a healthcare facility due to THE PERSON'S race/skin color

• Erratum

  In the article "Content validity of the scale Perceptions on Interpersonal Racial Discrimination in Brazilian Healthcare Facilities (Driss)", doi 10.1590/S0104-12902021200410, published on volume 30, no. 1 of 2021 of Saúde e Sociedade, on the first page:
  Where it reads:
  Ana Luisa Vilela Borges
  It should read:
  Ana Luíza Vilela Borges

Publication Dates

• Publication in this collection
  29 Mar 2021
• Date of issue
  2021

History

• Received
  24 May 2020
• Accepted
  09 Sept 2020