Child Well-Being Scales (CWBS): Psychometric Properties of the Portuguese Version

Abstract

There is a lack of validated instruments to evaluate child protection programs in Portugal. The present study analyzed the psychometric characteristics of the Child Well-Being Scales (CWBS), a multidimensional measure of potential situations of abuse and neglect, in 276 families at psychosocial risk. Measures of parental efficiency, satisfaction and parental symptomatology were also collected. Through exploratory factor analysis, a 3-factor solution, similar to the original version was identified, with high levels of internal consistency (α between 0.82-0.86) and good model fit (GFI = 0.97, RMSR = 0.06). Regarding construct validity, significant correlations were found that were in line with theoretical expectation. Our results support the utility of EBI as a guide to the assessment and decision-making regarding child protection measures.

Keywords:
factorial analysis; child well-being; psychometrics; child protection services

Resumo

Em Portugal, existe escassez de instrumentos aferidos para avaliar os programas de proteção da infância. Este estudo analisou as características psicométricas das Escalas de Bem-Estar Infantil (EBI), destinadas à avaliação multidimensional de potenciais situações de maltrato e negligência, em 276 famílias em risco psicossocial. Foram também recolhidas medidas de eficácia, satisfação e sintomatologia parental. A análise fatorial exploratória sugere uma estrutura com três fatores, similar à versão original, com boa consistência interna (α entre 0,82 e 0,86) e bons indicadores de ajustamento (GFI = 0,97, RMSR = 0,06). Relativamente à validade de construto, foram encontradas correlações significativas que correspondiam à expectativa teórica. Os resultados sustentam a aplicabilidade das EBI para a avaliação e tomada de decisão nas medidas de proteção da criança.

Palavras-chave:
análise fatorial; bem-estar infantil; psicometria; serviços de proteção de menores

Abstract

There is a lack of validated instruments to evaluate child protection programs in Portugal. The present study analyzed the psychometric characteristics of the Child Well-Being Scales (CWBS), a multidimensional measure of potential situations of abuse and neglect, in 276 families at psychosocial risk. Measures of parental efficiency, satisfaction and parental symptomatology were also collected. Through exploratory factor analysis, a 3-factor solution, similar to the original version was identified, with high levels of internal consistency (α between 0.82-0.86) and good model fit (GFI = 0.97, RMSR = 0.06). Regarding construct validity, significant correlations were found that were in line with theoretical expectation. Our results support the utility of EBI as a guide to the assessment and decision-making regarding child protection measures.

The evaluation and decision-making processes in Child Protection Services aim to determine whether a child should remain with their family or placed in the foster care system (James et al., 2019; Rodrigues et al., 2015), according to the intensity of maltreatment and the seriousness of the family risk. This process is complex, ambiguous and often riddled with errors and uncertainties is the basic pillar of the Child Protection Services intervention models (Grimaldi-Puyana et al., 2012; Ménendez, et al., 2016). As such, there is a continuous plea for the need to reduce false positives or false negatives, so that the evaluation outcomes can meet children’s needs adequately (López et al., 2015), thus allowing to intervene appropriately to prevent and protect children from maltreatment.

To improve Child Protection Services and to promote family preservation, it is key to ensure the efficacy of the evaluation and decision-making processes (Grimaldi et al., 2019). However, this evaluation has been mostly conducted based on practitioners’ observations, experience and clinical consensus (Ménendez et al., 2016; Pereira & Alarcão, 2015). To augment the reliability of such procedures, practitioners need tools that use a common language, one that can be understood and interpreted consistently by practitioners with different backgrounds. The evaluation must be structured, use different methods (e.g., questionnaires, observation, etc.) and use validated and appropriate assessment instruments (Bartelink et al., 2015; James et al., 2019; Leitão et al., 2020; Serbati et al., 2015).

There are a few self-report or practitioner report questionnaires that evaluate maltreatment risk in the family context, such as de Institutional Support in Cases of Child Maltreatment Instrument (I-APSI; Arredondo et al., 2017a), the Maltreatment Cases Recognition Instrument (I-REC; Arredondo et al., 2017b), the HOME Inventory (Home Observation for Measurement of the Environment; Caldwell & Bradley, 1984) and the Child Well-Being scales (Magura & Moses, 1986).

The Child Well-Being scales (Magura & Moses, 1986) are an interesting instrument due to their content validity and other psychometric characteristics. It is a multidimensional measure, with situations of potential maltreatment and neglect which was initially conceived to meet the assessment needs of Child Protective Services programs, although it is also possible to use it as a diagnostic tool for the assessment of individual cases. It consists of 43 items that evaluate the performance of the parenting role, family capabilities, family functioning and the child’s capabilities. Each item presents a detailed description of a behaviour or situation and includes four to six levels of specific answers, which range from adequacy to increasing levels of inadequacy. Each item is weighted according to the condition seriousness, with the maximum score per item being 100 (adequate level of child well-being).

Weightings were based on the opinions of a sample of hundreds of Child Protection Services front-line practitioners and stakeholders (Magura & Moses, 1986). The scales are applied based off the knowledge of the workers who are assigned to the case. This knowledge may be direct or obtained via reports, and the worker must have made at least one home visit. The assessed children must live in the family home at least part of their time. Not all scales are applicable to all ages.

From an exploratory factor analysis, Magura and Moses (1986) presented a 3-factor solution: Household Adequacy (10 items, items 2-10, 15), Parental Disposition (14 items, items 11, 16-29, 34, 40, which include parenting competences, difficulties, recognition of family problems, motivation and adult/child relationship) and Child Performance (4 items, items 37-39, 41), explaining 43% of the variance.

Since the original study, several researchers have used this instrument and reported on its psychometric properties. For instance, Gaudin et al. (1992) administered the Child Well-Being Scales to two different groups: 53 neglectful families and 80 non-neglectful families. The results indicated high levels of internal consistency (α = .92) and showed that the scale successfully differentiates family types and risk levels.

Similarly, in Canada the Child Well-Being Scales were used by 38 experts and common users with little variation in the inter-rater classification, thus corroborating the validity and sensitivity of the scale (Vézina & Pelletier, 1993). In Italy, a pre-post-test study was conducted using these scales to evaluate the effectiveness of an intervention program with 18 vulnerable families and 23 children (Serbati et al., 2015). The authors recommended that Child Well-Being Scales can be used to decrease subjectivity in the evaluation and to support the decision-making process.

A more recent study in Spain with 641 families at psychosocial risk demonstrated once more the discriminant power of the scale (Grimaldi et al., 2019). The Child Well-Being Scales explained 56% of the variance in the family outcomes in which children were placed in foster care and those in which children stayed with their family. Additionally, there was a child risk profile accuracy rate 88.12% as determined by the instrument (Grimaldi et al., 2019).

Notwithstanding these satisfactory data, the original factor structure has not been replicated by all the studies. In Spain, De Paúl and Arruabarrena (1999) conducted a validation study using exploratory factor analysis with 91 families and 231 children. They found a 3-factor solution, albeit with modifications regarding the item-subscale distribution: Parenting Care (10 scales on family functioning, items 1-5, 7, 9, 10, 13-16); Parental Disposition (14 scales on family functioning and child well-being, items 11, 19, 20, 21, 24-28) and Treatment of Child (4 scales on child well-being, consisting of items 11, 19, 20, 21, 24-28). However, the authors considered that the criticisms about the validity of this instrument could also be applied to any study that aims to assess child well-being, due to the lack of consensual theoretical support about it (De Paúl & Arruabarrena, 1999).

Despite this inconsistency, we may assert that the Child Well-Being Scales have substantial advantages, such as good content validity, acceptable internal consistency (α = .53 - .89), good test-retest reliability (Κ = .60 - .65), a satisfactory inter-rater agreement (Κ = .60) and good indicators of convergent and discriminant validity (Magura & Moses, 1986; Nasuti, 1998; Vézina & Bradet, 1990).

In Portugal there are no adaptation or validation studies published about the Child Well-Being scales, which constitutes a disadvantage for the scientific knowledge in this field and for the Child Protection Services workers (Rodrigues et al., 2015). To the present date, we are only aware of two pieces of research that have used the Child Well-Being scales with the Portuguese population. In 2017, Nunes and Ayala-Nunes studied 207 Portuguese families who had an active case in Family Preservation Families using the Child Well-Being scales with the aim of obtaining in-depth knowledge about the levels of parenting competence in this population and analysing the relations between sociodemographic variables, child well-being, perceived parenting efficacy and satisfaction with the parenting role. This study found significant positive associations between child well-being and the aforementioned parenting variables, pointing to the necessity of offering psychoeducational interventions with the parents as a means of promoting child well-being (Nunes & Ayala-Nunes, 2017). In the following year, a study was published in which the Child Well-Being scales were administered to 46 case managers concerning 249 parents in Portugal and Spain. This study aimed to investigate which were the determining factors of child well-being in Child Protection Services-referred children. The main findings highlighted economic hardship as a risk factor for children’s physical well-being, whereas parental satisfaction with the parenting role, parenting stress and an authoritarian parenting stile as the main thwarting factors for children’s emotional and academic well-being (Ayala-Nunes et al., 2018).

With the objective of filling the gap of a needed Portuguese validation and contributing for an adequate validation of this instrument, the present study intends to analyse the psychometric characteristics of the Child Well-Being Scales (Magura & Moses, 1986) in a sample of Portuguese at-risk families. Specifically, we aim to a) estimate the reliability and provide Portuguese norms (mean and standard deviation) for the Child Well-Being Scales; b) analysing their internal structure and c) describing the construct validity, by examining the associations between the Child Well-Being Scales scores and perceived parenting competences and parental mental health, which are relevant dimensions for child maltreatment.

Method

Participants

Forty-six practitioners from several Child Protection Service agencies in Algarve assessed 276 families. Participant selection criteria were as follows: 1) having an active case in Child Protection Services for family preservation reasons for at least three months; 2) not facing a family crisis (as determined by the case manager) during recruitment and data collection and 3) be the main carer of at least one underaged child.

Parents were on average 37.50 years old (SD = 8.30) and a low educational attainment: 27.37% had not completed the compulsory education level, 48.54% had completed the compulsory education level and only 19.34% and 4.74% had completed secondary and higher education, respectively. Regarding the work situation, 49.74% of participants were employed but mostly in low qualification jobs (71.59% worked as salespeople, janitors, etc.). Their children were between 3 months old and 18 years old (M = 10.61, SD = 4.69) and the majority were boys (61.71%)

Most families (84.13%) had a stable composition, with an average size of four people (M = 4.17, SD = 1.54) and two children and/or adolescents (M = 2.52, SD = 1.34). Over a third of the participants had a one-parent family (38.62%) and a quarter (23.81%) lived with at least a member of their extended family. Family income was unstable for 34.41% of the participants and they earned on average 924.87 euros per month (SD = 578.95), with 34.76% of the families receiving some type of financial support benefit.

Case managers were mostly women (90%) with ages ranging between 30 and 40 years (66%). Regarding their professional category, the majority were psychologists (48%) but there were also educators (29%), social workers (19%) and legal experts (5%). Most case workers had 5 or more years of experience working with families (62%).

Instruments

Child well-being scales

Described above.

Perceived parenting competence (PSOC)

We used Nunes et al.’s (2016) Portuguese version of the Parenting Sense of Competence scale. This scale, originally developed by Gibaud-Wallston and Wandersman (1978) and later adapted by Johnston and Mash (1989), assesses one’s perceived competence as a parent through two dimensions: efficacy and satisfaction. The Efficacy dimension has seven items which measure to which extent the parent feels competent in their parenting role (e.g., “Despite being difficult, I already know how to influence children”). The Satisfaction dimension has nine items and aims to determine the degree to which the caregiver feels satisfied with their parental role (e.g., “Even though being a parent could be rewarding, I am frustrated now while my child is at his/her present age”). There are six response options ranging from 1 (“no, strongly disagree”) to 6 (“yes, strongly agree”). The minimum and maximum scores for the Efficacy dimension are 7 and 42, respectively, whereas for the Satisfaction dimension scores can range from 9 to 54. Higher scores correspond to a higher perception of efficacy and satisfaction with parenting. The reliability index of the scale in the present study was ω = .78 for Efficacy and ω =.75 for Satisfaction.

General Health Questionnaire (GHQ 28)

We used the Portuguese version (Pais-Ribeiro & Antunes, 2003) of the short form of the General Health Questionnaire (Goldberg & Williams, 1996). This is a 28-item questionnaire that seeks to detect current psychological problems from the respondent’s reporting of symptoms they have experienced during the last three weeks. These symptoms are grouped into four subscales with seven items each: Somatic symptoms (e.g., “Have you recently been getting any pains in your head?”), anxiety and insomnia (e.g., “Have you recently had difficulty staying asleep once you are off?”), social dysfunction (e.g., “Have you recently been satisfied with the way you’ve carried out your task?”) and severe depression (e.g., “Have you recently felt that life isn’t worth living?”). Each item has four response options which vary according to the question, with higher scores representing a more intense symptom experience. However, the subscales neither constitute independent medical categories, nor do they correspond exactly with psychiatric diagnoses. The reliability index of the scale in this study was ω = .92.

Procedures

We requested permission to translate and validate the Child Well-being Scales to Portuguese to the first author of the scale (Magura & Moses, 1986, 1987) and to the Child Welfare League of America (Washington, DC). The initial translation from English to Portuguese was made by the first two authors of this study, ensuring that practitioners correctly understood the meaning of the items. The questionnaire was then translated again to English by a native with considerable professional experience in the translation of scientific texts in Psychology. Cultural adaptation was especially considered, while ensuring the clarity, common language use and conceptual equivalence of the scale.

After establishing the collaboration protocols with Child Protection Services of Algarve, practitioners of the collaborating institutions selected the parents who met the selection criteria and invited them to participate in the study. Afterwards, nine interviewers - who had received specific training to apply the instruments - travelled to the institution to interview the parents who had agreed to participate.

The 46 case manager practitioners completed the Child Welfare Scales after attending a five-hour training session. Before completing the questionnaires, participants were informed about the study aims, the non-compensatory nature of their participation, the anonymous and confidential nature of their answers and the possibility of withdrawing the participation of the study at any moment without consequence.

The PSOC, GHQ-28 and sociodemographic questionnaire were administered via an individual interview lasting approximately 30 minutes.

Statistical analyses

We IBM SPSS 20 and FACTOR 9.2 (Lorenzo-Seva & Ferrando, 2006) to code and analyse the data. Firstly, we examined the existence of multivariate extreme cases through the calculation of Mahalanobis distance (Tabachnick & Fidell, 2019) and computed the descriptive statistics of the original scales. Afterwards, we analysed the normality of the univariate distribution of the items, considering the asymmetry and kurtosis indices (values with ranges between the ±2 intervals), as recommended by Bandalos and Finney (2010). After this procedure, we analysed the discriminant capacity of each item through the corrected correlation coefficient between the item score and the scale total (> ,25) and the reliability if the item was deleted (lower than the global dimension) (Ferrando & Anguiano-Carrasco, 2010).

The instrument dimensionality was analysed through factor analysis (FA) with the program FACTOR Vs. 9.2 (Lorenzo-Seva & Ferrando, 2006). We verified the normality and linearity assumptions between each pair of variables. We calculated the Pearson correlation matrix and estimated the reliability through the ordinal alpha coefficient. As the estimation method we used the maximum likelihood estimation and an oblique rotation, through the normalised direct Oblimin method. The matrix factorability was established through high values in the Kaiser-Meyer-Olkin sample adequacy measure and a significant value in Barlett’s sphericity test (Carretero-Dios & Pérez, 2005; Tabachnick & Fidell, 2019).

To decide the number of factors to retain, we considered the following criteria: eigenvalues values > 1, a minimum of three variables per factor (Ferrando & Anguiano-Carrasco, 2010); a parallel analysis with the optimal implementation of Timmerman and Lorenzo-Seva (2011); interpretability and theoretical relevance. We retained the items with a configuration coefficient higher than .30 in only of the factors. To evaluate goodness of adjustment of the factorial solution, we analysed the simplicity indices S and LS (close to 2), as well as the GFI indices (recommended > .95) and RMSR (close to 0).

To assess the internal consistency of the obtained factors we calculated the ordinal alpha indices and descriptive statistics. Criterion validity of the proposed version was analysed through the Pearson correlation indexes of the obtained factors with the scores of the perceived parenting competences and mental health.

Results

Initial Descriptive Analysis

From the Mahalanobis distance calculation we identified 4 multivariate extreme cases (1.45%), which were deleted from subsequent analyses. According to the authors’ original version, the scales presented the following values (Table 1).

Table 1
Descriptive statistics of the original Child Well-Being scales (N = 272)

	CWBS total	Household Adequacy	Parental Disposition	Child Performance
Mean	84.85	90.91	78.73	82.12
Standard deviation	8.61	11.03	11.38	17.28
Range	56.89 - 100	44.70 - 100	40.62 - 100	37.75 - 100
Asymmetry	-0.48	-1.49	-0.29	-0.81
Kurtosis	0.02	1.76	0.21	-0.49
α	.89	.86	.85	.82

All dimensions were significantly and positively correlated, except the Household Adequacy with Child Performance (Table 2). Scores were lower in Parental Disposition than in Household Adequacy (t ₍₂₇₁₎ = 18,12; p < ,001) and in Child Performance (t ₍₂₇₁₎ = -3,41; p < ,001). Next, we will proceed to the exploratory factor analysis.

Table 2
Correlation indices between the dimension of the original version of CWBS

	1	2	3	4
1. Household Adequacy	-	.51***	.02	.73***
2. Parental Disposition		-	.41***	.91***
3. Child Performance			-	.52***
4. CWBS total				-

• *p < .05; ** p < .01; *** p < .001

Factorial Structure

Items 17, 8, 23, 33, 35, 36 and 43 were excluded due to the high number of missing cases. Due to the high indices of asymmetry and kurtosis (±2), we deleted items 1, 3, 8, 9, 30, 31 and 32 from subsequent analysis (see Table 3). The remaining items had high correlations with the rest of the scale (> .25) and there were no improvements in the dimension reliability if any of them was excluded, thus they were kept for subsequent analyses.

Table 3
Analysis of the discriminant capacity of the original items (N = 272)

	M	SD	Asymmetry	Kurtosis	Corrected r item-total corrigida	α if item deleted
Item 1	90.37	16.95	-2.27	5.66	-	-
Item 2	84.85	20.46	-1.06	0.15	.64	.80
Item 3	94.62	10.24	-2.40	7.38	-	-
Item 4	90.23	17.14	-1.71	1.72	.67	.80
Item 5	92.43	13.10	-1.75	1.82	.75	.80
Item 6	92.61	13.61	-1.76	1.99	.47	.83
Item 7	86.64	21.38	-1.51	1.34	.67	.80
Item 8	97.00	8.06	-3.12	8.67	-	-
Item 9	96.20	10.25	-3.23	10.37	-	-
Item 10	84.08	27.35	-1.19	-0.48	.55	.83
Item 15	89.50	15.15	-1.33	0.37	.52	.82
α sub-total						.83
Item 11	80.04	21.27	-0.68	-0.09	.44	.86
Item 12	86.15	17.89	-1.16	0.90	.47	.86
Item 13	72.20	20.23	-0.03	-1.26	.53	.86
Item 14	88.11	16.36	-1.34	1.47	.44	.86
Item 16	72.92	25.62	-0.51	-0.35	.50	.86
Item 19	64.90	21.97	0.88	-1.00	.55	.86
Item 20	71.75	17.44	0.56	-0.63	.68	.85
Item 21	75.66	21.86	0.18	-1.89	.54	.86
Item 24	79.14	18.93	-0.30	-0.81	.59	.85
Item 25	84.59	12.00	0.34	-1.45	.60	.86
Item 26	78.63	15.25	-0.44	-0.12	.69	.85
Item 27	78.63	14.72	-0.52	-0.66	.67	.85
Item 28	79.07	12.79	-0.44	1.16	.62	.86
Item 29	93.12	19.76	-1.88	1.89	.34	.87
Item 30	97.58	9.74	-5.06	30.17	-	-
Item 31	98.53	5.13	-11.52	165.17	-	-
Item 32	97.93	10.26	-4.96	23.98	-	-
Item 34	89.22	21.10	-1.57	0.89	.34	.87
Item 40	85.31	18.91	-1.02	0.26	.34	.87
α sub-total						.87
Item 37	89.21	17.39	-1.35	0.66	.55	.82
Item 38	79.18	20.76	-0.39	-1.25	.76	.76
Item 39	83.26	24.25	-1.00	-0.78	.67	.78
Item 41	78.91	25.28	-0.48	-1.50	.65	.79
Item 42	79.71	20.46	-0.35	-1.12	.52	.82
α sub-total						.83

The matrix factorability was established by obtaining an acceptable value in the Kaiser-Meyer-Olkin test (KMO = .89) and a significant value in Bartlett’s sphericity test (χ ²(378) = 3693.3; p < .000). Factor analysis provided a three-factor solution with eigenvalues values higher than 1, which contributed to explain a variance of 49.52%, and the Parallel Analysis results recommended retaining the same three factors. The first factor explained 30.64% of the variance (λ = 8.58) and included items 2, 4, 5, 6, 7, 10, 11, 12, 14, 15 and 16, almost completely replicating the original dimension of Household Adequacy, besides a few items referring to parents’ ability to care for their children, which we labelled as Parenting Care. The second factor explained a variance of 12.61% (λ = 3.53) and retained items 13, 19, 20, 21, 24, 25, 26, 28, 29, 34 and 40, which are similar to the Parental Disposition of the original version. The third factor explained 6.27% (λ = 1.76) of the variance and comprised items 37, 38, 39, 41 and 42, almost totally replicating the Child Performance dimension of the original version. Goodness-of-fit indices for this solution were satisfactory (GFI = .97; RMSR = .06; S = .99; LS = .52). Items 13 and 28 saturated in two factors of the standard matrix with values ≥ .40 (Table 4) and were thus removed from subsequent analyses.

Table 4
Results of confirmatory PCA

	Standard matrix			Structural matrix
	F1	F2	F3	F1	F2	F3
Item 2			.62	.46		.71
Item 4			.72			.72
Item 5			.79			.77
Item 6			.61			.54
Item 7			.71			.72
Item 10			.63			.62
Item 11			.35			.40
Item 12			.55	.32		.59
Item 13		-.37	.32	.49	-.49	.44
Item 14			.44			.47
Item 15			.51	.39		.59
Item 16			.40	.42		.50
Item 19	.49			.57		.39
Item 20	.56			.69	-.30	.51
Item 21	.53			.59		.37
Item 24	.65			.71	-.34	.36
Item 25	.65			.69	-.33	.33
Item 26	.70			.78	-.40	.39
Item 27	.54			.72	-.52	.40
Item 28	.36		.35	.59	-.41	.51
Item 29	.41			.38
Item 34	.43			.38
Item 37		-.62			-.62
Item 38		-.88		.33	-.86
Item 39		-.82			-.78
Item 40	.31			.38	-.37
Item 41		-.69		.35	-.72
Item 42		-.44		.40	-.55

• Note: F = Fator

The descriptive analysis of the factors showed an mean of 85.68 (SD = 13.08) for Parenting Care, 79.92 (SD = 12.09) for Parental Disposition and 81.91 (SD = 16.73) for Child Performance. We present the correlation indices between the three factors and their reliability coefficients (see table 5).

Table 5
Descriptive statistics, reliability indices and correlations between the CWBS factors

	PC	PD	CP	CWBS
PC	-	.53***	.16*	.81***
PD		-	.42***	.87***
CP			-	.56***
CWBS				-
M (SD)	85.68 (13.08)	79.92 (12.09)	81.91 (16.73)	82.74(10.32)
Range	37.18 - 100	41.50 - 100	39.60 - 100	44.88 -100
α	.86	.84	.82	.89
ω	.88	.86	.84	.89
Ordinal alpha	.88	.86	.83	.90

• Note: PC= Parenting Care, PD= Parental Disposition, CP = Child Performance. CWBS = Child Well-Being Scales. M = Mean, SD = Standard deviation, α = Crombach’s alpha, ω = Omega.
• *p < .05; ** p < .01; *** p < .001

Construct validity

As we can observe in Table 6, there is a negative and significant correlation between all the Child Well-Being Scales dimensions and parental mental health. However, we only saw significant and positive relations between perceived parenting efficacy and Child Performance. As for the satisfaction with the parenting role, it was positively and significantly correlated with Parental Disposition and Child Performance.

Table 6
Correlations between child well-being, perceived parenting competences (N = 276) and parental mental health (N = 111)

	Parental perceived efficacy	Satisfaction with parenting role	Mental Health
Parenting Care	-.07	.05	-.31**
Parental Disposition	.11	.19**	-.32**
Child Performance	.19**	.25***	-.29**
CWBS Total	.07	.19**	-.42***

• *p < .05; ** p < .01; *** p < .001

Discussion

The main objective of the present study was to respond to the scarcity of adapted instruments in Portugal, especially of evaluations tools available for Child Protection Services (Rodrigues et al., 2015). As such, we intended to adapt and validate the Child Well-Being Scales for the Portuguese population, with a sample of at-risk families.

The findings of this study replicated some of the psychometric characteristics of the original development study and other studies with at-risk populations. The obtained factor solution retained three-factor, in accordance with previous research (De Paúl & Arruabarrena, 1999; Magura & Moses, 1986; Vézina & Pelletier, 1993) and explained a higher proportion of the variance (50%) than the original study (43%) with good reliability indices in all subscales (α > ,82).

The first factor (Parenting Care) retained most of the items of the original version, named Household Adequacy (7 out of 10 items) and included a few of the items related to the parents’ ability to care for their children (4 items), as presented in De Paúl and Arruabarrena’s (1999) study. Although this factor does not fully replicate the proposed dimension in the original development study, it presents high levels of internal consistency which are higher than the ones found in both the original and the Spanish version.

The second factor (Parental Disposition) was similar to the original version and retained most of the items from the original scale (10 out of 14). Finally, the third factor almost completely mirrored the original Child Performance scale.

Regarding the mean values of each of the subscales obtained by the present sample, the scores were slightly higher compared to other validation studies in Spain and Italy (Grimaldi et al., 2019; Serbati et al., 2015). This suggests that the reference scores for Portuguese families tend to be higher than those observed in other Southern European countries.

However, compared to the Spanish validation by De Paúl and Arruabarrena (1999), the subscale Child Performance in our study had a lower score (M = 81.91) than the Spanish sample of at-risk families (M = 92.0). This lower score in Child Performance may be particularly relevant as it could indicate that for at-risk Portuguese children the largest deleterious effect of their context is seen in indicators that are comprised in this dimension, such as a poor academic performance and problematic behaviours. Therefore, we suggest that when families are assessed, practitioners pay special attention to this dimension. According to Serbati et al. (2015), this was also the subscale in which largest short-term benefits were seen after family preservation interventions.

We found evidence for construct validity of the Child Well-Being Scales, as they were related to other relevant dimensions such as parenting sense of competence and parental mental health. The positive association between satisfaction with the parenting role and the child well-being dimensions had been documented before in other studies using the same measures, such as the one by Nunes e Ayala-Nunes (2017). Similarly, the relation between parental perceived efficacy and Child Performance had been a highlighted aspect in recent research (e.g., Grimaldi et al., 2019), in which the perceived inefficacy in the parenting role was a strong predictor of child well-being.

Thus, notwithstanding the heterogeneity of at-risk families’ difficulties, the associations that we found suggest indicators that have been systematically proven (Grimaldi et al., 2019; Ménendez et al., 2016) and should be considered when evaluating and intervening in the child protection and family preservation realm.

Considering this picture, the psychometric analyses allow us to affirm that the Child Well-Being Scales are a reliable and valid instrument to assess potential maltreatment and neglect cases. It is a tool that allows to structure and systematise the practitioners’ observations about the basic characteristics of the home, parental and child’s functioning. This characteristic is especially important, since when evaluating maltreatment and neglect it is key to achieve homogeneity, both in the variables that must be observed and in the description and categorisation of the levels of severity of these variables (Serbati et al., 2015). By standardising the Child Well-Being scales allow to overcome the errors that are commonly attributed to technicians’ decision-making process, which is often made based on their subjectiveness and clinical sense (Grimaldi-Puyana et al., 2012; López et al., 2015).

In a realm in which decisions can have serious and dramatic consequences for families and children, we cannot rely on intuition, as it is often contaminated by individual and cultural biases. Thus, the validation of the Child Well-Being scales offers Child Protection Services workers the possibility of using a common language and basing their work on the same principles, so that their decisions are more reliable and consistent.

Nevertheless, the favourable contribution of the validation, the reduced sample size and limited geographic distribution of the participants, circumscribed to one region of the country are disadvantages that limit the generalisability of the findings. In addition, there are other limitations linked to the cross-sectional design of the study, such as the absence of test-retest analysis and the inclusion of only one sample type, which made it impossible to conduct inter-group comparisons.

Future research should aim to replicate this study in a more representative sample that includes distinct groups (e.g., families with different levels of psychosocial risk), as a means to obtain a higher external validity and information about the sensitivity of the scale.

References

• Arredondo, V., Saavedra, C., & Guerra, C. (2017a). Evaluación de un instrumento de apoyo institucional en casos de maltrato infantil. Revista Evaluar, 17(2), 165-176. https://doi.org/10.35670/1667-4545.v17.n2.18729
  » https://doi.org/10.35670/1667-4545.v17.n2.18729
• Arredondo, V., Saavedra, C., & Guerra, C. (2017b). Análisis psicométrico preliminar de un instrumento para evaluar el reconocimiento de situaciones de maltrato infantil (I-REC) en Chile. Revista de Psicología Clínica Con Niños y Adolescentes, 4(2), 111-117. http://www.revistapcna.com/sites/default/files/16-28.pdf
  » http://www.revistapcna.com/sites/default/files/16-28.pdf
• Ayala-Nunes, L., Jiménez, L., Jesus, S., Nunes, C., & Hidalgo, V. (2018). An ecological model of well-being in child welfare referred children. Social Indicators Research, 140(2), 811-836. https://doi.org/10.1007/s11205-017-1807-x
  » https://doi.org/10.1007/s11205-017-1807-x
• Bandalos, D. L., & Finney, S. J. (2010). Factor Analysis: Exploratory and Confirmatory. In G. R. Hancock y R. O. Mueller (Eds.), Reviewer’s guide to quantitative methods (pp. 98-122). Routledge.
• Bartelink, C., van Yperen, T. A., & ten Berge, I. J. (2015). Deciding on child maltreatment: A literature review on methods that improve decision-making. Child Abuse and Neglect, 49, 142-153. https://doi.org/10.1016/j.chiabu.2015.07.002
  » https://doi.org/10.1016/j.chiabu.2015.07.002
• Caldwell, B. M., & Bradley, R. H. (1984). Home observation for measurement of the environment University of Arkansas.
• Carretero-Dios, H. & Pérez, C. (2005). Normas para el desarrollo y revisión de estudios instrumentales. International Journal of Clinical and Health Psychology, 5, 521-551. https://www.redalyc.org/articulo.oa?id=337/33705307
  » https://www.redalyc.org/articulo.oa?id=337/33705307
• De Paúl, J., & Arruabarrena, M. I. (1999). Escalas de Bienestar Infantil de Magura y Moses. Un primer estudio para la validación de la versión española. Psychosocial Intervention, 8(1), 89-108. http://www.copmadrid.org/webcopm/publicaciones/social/47562.pdf
  » http://www.copmadrid.org/webcopm/publicaciones/social/47562.pdf
• Ferrando, J. P., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología. Papeles del Psicologo, 31(1), 18-33. http://www.papelesdelpsicologo.es/contenido?num=1137
  » http://www.papelesdelpsicologo.es/contenido?num=1137
• Gaudín, J. M., Polansky, N. A., & Kilpatrick, A. C. (1992). The Child Well-Being Scales: A field trial. Child Welfare, 71(4), 319-328.
• Gibaud-Wallston, J., & Wandersmann, L. P. (1978). Development and utility of the Parenting Sense of Competence Scale John F. Kennedy center for research on education and human development.
• Goldberg, D., & Williams, P. (1996). Cuestionario de salud general GHQ (General Health Questionnaire). Guía para el usuario de las distintas versiones Masson.
• Grimaldi-Puyana, V., Garrido-Fernández, M., & Jiménez-Morago, J. (2012). Perfiles de riesgo infantil y niveles de intervención con familias usuarias del Sistema Público Servicios Sociales. Anales de Psicología, 28(2), 515-523. https://doi.org/10.6018/analesps.28.2.148751
  » https://doi.org/10.6018/analesps.28.2.148751
• Grimaldi, V., Pérez-Padilla, J., Garrido, M. Á., & Lorence, B. (2019). Assessment and decision-making in child protective services: Risk situations kept-at-home versus out-of-home care. Child Indicators Research, 12(5), 1611-1628. https://doi.org/10.1007/s12187-018-9600-1
  » https://doi.org/10.1007/s12187-018-9600-1
• James, A., McLeod, J., Hendy, S., Marks, K., Rusu, D., Nik, S., & Plank, M. J. (2019). Using family network data in child protection services. PLoS ONE, 14(10), e0224554. https://doi.org/10.1371/journal.pone.0224554
  » https://doi.org/10.1371/journal.pone.0224554
• Johnston, C., & Mash, E. J. (1989). A measure of parenting satisfaction and efficacy. Journal of Clinical Child Psychology, 18(2), 167-175. https://doi.org/10.1207/s15374424jccp1802_8
  » https://doi.org/10.1207/s15374424jccp1802_8
• Leitão, I. B., Dias, A., Tristão, K., Ronchi, J., & Avellar, L. (2020). Dez anos de um CAPSi: Comparação da caracterização de usuários atendidos. Psicologia USP, 31, e190011. https://doi.org/10.1590/0103-6564e190011
  » https://doi.org/10.1590/0103-6564e190011
• López, M., Fluke, J. D., Benbenishty, R., & Knorth, E. J. (2015). Commentary on decision-making and judgments in child maltreatment prevention and response: An overview. Child Abuse and Neglect, 49(1), 1-11. https://doi.org/10.1016/j.chiabu.2015.08.013
  » https://doi.org/10.1016/j.chiabu.2015.08.013
• Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: A computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38(1), 88-91. https://doi.org/10.3758/BF03192753
  » https://doi.org/10.3758/BF03192753
• Magura, S., & Moses, B. S. (1986). Outcome measures for child welfare services: Theory and applications Child Welfare League of America.
• Magura, S., & Moses, B. S. (1987). Outcome measures for child welfare services: Child well‐being scales and rating form Child Welfare League of America.
• Ménendez, S. A., Hidalgo, M. V. G., Lorence, B. L., & Pérez, J. P. (2016). Assessing the level of risk of families supported by child and family protection services: Practitioners and mothers as informants. Journal of Social Work, 16(5), 595-609. https://doi.org/10.1177/1468017315583174
  » https://doi.org/10.1177/1468017315583174
• Nasuti, J. P. (1998). Risk assessment in child protective services: Challenges in measuring child well-being. Journal of Family Social Work, 3(1), 55-70. https://doi.org/10.1300/J039v03n01_05
  » https://doi.org/10.1300/J039v03n01_05
• Nunes, C., & Ayala-Nunes, L. (2017). Parenting sense of competence in at psychosocial risk families and child well-being. Bordon, 69(1), 155-168. https://doi.org/10.13042/Bordon.2016.48589
  » https://doi.org/10.13042/Bordon.2016.48589
• Nunes, C., Jiménez, L., Menéndez, S., Ayala‐Nunes, L., & Hidalgo, V. (2016). Psychometric properties of an adapted version of the parental sense of competence (PSOC) scale for Portuguese at‐risk parents. Child & Family Social Work, 21(4), 433-441. https://doi.org/10.1111/cfs.12159
  » https://doi.org/10.1111/cfs.12159
• Pais-Ribeiro, J., & Antunes, S. (2003). Contribuição para o estudo de adaptação do Questionário de Saúde Geral de 28 Itens (General Health Questionnaire-GHQ28). Revista Portuguesa de Psicossomática, 5(1), 37-45. https://www.redalyc.org/articulo.oa?id=28750105
  » https://www.redalyc.org/articulo.oa?id=28750105
• Pereira, D., & Alarcão, M. (2015). Guia de avaliação das capacidades parentais: Estudo de validade ecológica. Psicologia: Teoria e Pesquisa, 31(2), 203-212. https://periodicos.unb.br/index.php/revistaptp/article/view/18124
  » https://periodicos.unb.br/index.php/revistaptp/article/view/18124
• Rodrigues, L., Calheiros, M., & Pereira, C. (2015). The decision of out-of-home placement in residential care after parental neglect: Empirically testing a psychosocial model. Child Abuse and Neglect, 49, 35-49. https://doi.org/10.1016/j.chiabu.2015.03.014
  » https://doi.org/10.1016/j.chiabu.2015.03.014
• Serbati, S., Pivetti, M., & Gioga, G. (2015). Child Well-Being Scales (CWBS) in the assessment of families and children in home-care intervention: An empirical study. Child and Family Social Work, 20(4), 446-458. https://doi.org/10.1111/cfs.12094
  » https://doi.org/10.1111/cfs.12094
• Tabachnick, B. G. & Fidell, L. S. (2019). Using multivariate statistics (7^th ed.). Pearson Education.
• Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-220. https://doi.org/10.1037/a0023353
  » https://doi.org/10.1037/a0023353
• Vézina, A., & Pelletier, D. (1993). Un instrument de mesure pour faciliter l’évaluation et l’intervention en protection de la jeunesse. Service Social, 42(2), 63-81. https://doi.org/10.7202/706618ar
  » https://doi.org/10.7202/706618ar
• Vézina, A., & Bradet, R. (1990). Validation québécoise de l’inventaire concernant le bien-être de l’enfant en relation avec l’exercice des responsabilités parentales. Rapport de recherche Centre de recherche sur les services communautaires: Université Laval.

Publication Dates

• Publication in this collection
  28 Oct 2022
• Date of issue
  2022

History

• Received
  23 Dec 2020
• Accepted
  30 Mar 2021