Abstract
Objectives: There is a critical need to monitor the development of children around the world, and in Brazil, this need is substantial since there is a paucity of assessment tools. This study aimed to describe the design and provide evidence of reliability and validity for the short version of the Dimensional Inventory for Child Development Assessment (IDADI-short).
Methods: A sample of 1,865 biological mothers of children aged 4-72 months (M = 34.8, SD = 20.20) completed the IDADI to assess Cognitive, socio-emotional, Expressive, and Receptive Language and Communication, Fine and Gross Motor, and Adaptive Behavior development. The psychometric properties of a total of 118 subscales of IDADI were obtained and the IDADI-short age-specific scores were correlated with the original inventory, and criteria variables such as neurodevelopment diagnosis, socioeconomic status, and sex.
Results: Item Response Theory analysis, Cronbach’s Alpha, and McDonald’s Omega indicated excellent internal consistency and optimal participant discrimination after minor alterations. IDADI-short scores were strongly associated with the original inventory, with high sensibility and specificity precision for developmental delays. Significant associations with relevant criteria variables were also observed.
Conclusion: Findings support the use of IDADI-short as a parental measure of young children’s development.
KEYWORDS Child development; Assessment; Validation; Item response theory
Introduction
Developmental researchers have long recognized the complex and multidimensional perspectives needed to study human developmental phenomena.1 Child development makes the investigation more complex as changes are observed at a surprising rate.2 This phenomenon is characterized by remarkable and rapid bio-behavioral changes3 wherein the fast physical and neural development, social relations, and the environment interact favoring great advances in the child’s capacity during this sensitive period of development.4,5
Early healthy development is an important determinant of long-term mental and physical outcomes. There is substantial evidence supporting the importance of early identification of developmental delays and intervention for children’s developmental health and well-being.6,7 This phenomenon is related to the impact that early life experiences and genetic dispositions have on cognitive, social, behavioral, and emotional development.4,8 As such, it is essential to assess the development of children with and without developmental disabilities, as evidence has shown that ‘‘early intervention might be even more important for children with developmental disabilities than for typically developing children, because learning is cumulative, and barriers to healthy development early in life may impede development in subsequent stages’’.2
In early childhood, negative outcomes might be mitigated when delays or development problems are detected and treated early.7,8 Unfortunately, studies have shown that primary care professionals detect less than one-third of delays, and it is estimated that clinical assessments can detect only 30% of children with delays before they reach school age.9 Accordingly, the use of screeners of developmental milestones has been increasingly recommended as an important resource for improving the quality of evaluations in public health services.10 Although health professionals emphasize the screening of young children’s development, there is a paucity of standardized assessment tools with normative data for the Brazilian context.11 The scarcity is more critical when considering the population of children below three years of age living in Brazil.11 Recently, the third edition of the Bayley Scale of Infant Development (BSID)12 started to be commercialized, but there are some concerns regarding the adaptation process such as the use of outdated North-American norms, and the lack of a cultural adaptation study. The high cost associated with this tool and the requirement of highly trained professionals is another limitation that has precluded its use on a large scale. The under detection of children with developmental delays is thus a particularly critical concern in Brazil.
Given the paucity of instruments designed for use in the Brazilian context, there is a clear demand for an instrument that can benefit both the research and clinical communities. For this, the Dimensional Inventory for Child Development Assessment (IDADI; Inventário Dimensional de Avaliação do Desenvolvimento Infantil) was developed by Silva, de Mendonça Filho, and Bandeira.13 An extensive, multi-phased process was taken to develop items for the IDADI, wherein IDADI’s items were either inspired from existing instruments and modified for the parental report, or newly written specifically for the Brazilian context. This included content scrutiny by 18 subject matter experts, semantic analysis using a four-session focus group with seven mothers of low socioeconomic status, which were purposefully selected to check item comprehension, and a pilot study with 12 mothers who answered the full set of IDADI items.14 IDADI also underwent various studies to obtain validity and reliability evidence.15,16
IDADI administration takes around 30-40 min, and psychometric analyses indicated that the 435 items cover a great continuum of developmental abilities, indicating that IDADI surveys more than just developmental milestones. Due to the lengthy testing time, the need for a short version that could favor the identification of children who need special attention is crucial for effective monitoring in primary care settings. Thus, the objective of the present study was to empirically derive a short version of the Dimensional Inventory for Child Development Assessment (IDADI-B) and gather reliability and validity evidence.
Method
Participants and procedures
A convenience sample of 1865 biological mothers of children aged 4-72 months (M = 34.8, SD = 20.20) took part in the study. Approximately one-third of participants were recruited from public and private institutions of education, university clinical service centers, and health service centers (public, private, and philanthropic sectors) located across different regions of Brazil. These participants completed paper-and-pencil versions of the IDADI. The remaining participants (66.8%) were recruited via the internet using a media release coordinated by the research team. Internet participants completed an online questionnaire on the SurveyMonkey® platform. Only cases with complete data from the Sociodemographic and Clinical Questionnaire of the online data collection were considered. Both types of data collection included the same set of items and instruments. Mothers and their children were eligible for the study if the mother and child lived together and had daily contact.
Most participants were from the South region of Brazil (67.7%), followed by the Southeast (22.5%), Northeast (4.7%), Midwest (3%), and North (1.6%). The majority of mothers (79%) self-identified as white, 15.4% brown, 4% black, 1.5% yellow (i.e., East Asian), and 0.2% indigenous. Children’s sex distribution was approximately equal with a slight majority identified as male (51.1%). A small number of cases (approximately 5%) reported a developmental diagnosis (2% autism spectrum disorder, 0.6% Down syndrome, 0.5% cerebral palsy, 0.5% developmental coordination disorder, 0.2% intellectual disability, and 0.8% had another type of diagnosis). In terms of economic status, 37.8% of the women had a family income greater than five times the minimum wage and only 4.4% had an income of less than one minimum wage per month. The majority (69.0%) of mothers were involved in paid work (Table 1).
All participants provided informed consent, which complied with the Brazilian National Health Council guideline 466/12 as well as earlier guidelines. The study was approved by the research ethics committee of the Institute of Psychology from the Federal University of Rio Grande do Sul (UFRGS; CAAE: 45991815.5.0000.5334).
Instruments
Dimensional inventory for child development assessment The IDADI is a multidimensional inventory that assesses Cognitive, Communication, and Language (Expressive and Receptive), Motor (Gross and Fine), Socio-emotional, and Adaptive Behavior. The answer alternatives for each item are given as follows: 0 = not yet, 1 = sometimes and 2 = yes. Items of each developmental domain are divided across 17 age-bands (with exception of the Adaptive Behavior domain that covers 16 age-bands), yielding a total of 118 subscales with a varying number of items depending on the domain and the age-band considered. The detailed item information for each domain and age-group is available in Supplementary Tables S8-S14. The items of each original subscale were selected according to their respective levels of difficulty and precision, which were defined by Item Response Theory (IRT) analyses.
Sociodemographic (SES) and clinical questionnaire This questionnaire aimed to provide data on variables identified by the literature as being related to the children’s development. Examples of the information that were collected via mother report include: 1) family demographics (i.e., social, cultural, economic, parental education), 2) mother’s health during pregnancy, 3) pregnancy and birth conditions, and the child’s early development, and 4) child’s environment and interactions with caregivers.
Data analysis plan
Intending to empirically derive a short version of IDADI, data analysis consisted of two stages. First, psychometric characteristics of each of the 17 age group subscales from each IDADI domain were investigated using the rating scale model by Andrich,17 an extension of the 1-parameter (Rasch), IRT model.18 The purpose of this step was to select the seven most informative items considering the total variation of the latent trait for each age group. The information function is a quantification of how well an individual item and the item set (test or subscales) estimate the person location parameters.18 Thus, it provides an indication of the items that can estimate child development with the least error. The estimation of the cumulative item information considered the whole range of abilities of each age group. The choice of seven items per age group/domain was based on the average number of items used in gold-standard screening instruments.19,20
Second, an IRT analysis with the selected items was performed to investigate the fit of the items to the short format. The infit mean-square, a statistic with an approximate chi-square distribution that indicates the discrepancy between the model-predicted probabilities and empirical responses, was used to determine the item fit to the measurement model.21 Items were considered adequate if infit values were between 0.5 and 1.5.21 We also considered Cronbach’s Alpha and McDonald’s Omega22 to assess evidence of subscale-level reliability. To account for the categorical nature of the item responses, reliability was estimated from the polychoric correlation. Reliability was considered adequate if values were above 0.7.23 If the criteria for infit and reliability were not met, the eighth-most informative item replaced the item with fit problems or with less contribution to the subscale’s reliability. If the criteria adopted were still not met, the next item was selected.
Correlations between IDADI-B and its original version were computed, as well as sensitivity and specificity receiver operating characteristic (RoC) curves for developmental delay classifications. This allowed comparing the true positive rate (sensitivity) and true negative rate (specificity) of IDADI-B screening potential in relation to the original inventory. We standardized IDADI-B scores using means and standard deviations of the sample of children without developmental diagnosis or health problems. Children screened positive for development delay if an IDADI-B z-score below-1 SD was obtained. Multiple regression analyses were performed to gather additional validity evidence based on diagnosis, maternal education, family income, and sex.
Results
The seven domains and 17 age-groups of IDADI resulted in the analysis of 118 subscales. Data from some questionnaires were analyzed in a single run because the items on these questionnaires are exactly the same.
Subscales of the Cognitive domain presented a Cronbach’s Alpha and McDonald’s Omega that varied from 0.75 to 0.93, and 0.85 to 0.97 respectively, indicating good to excellent consistency values (Tables 2 and S1). Moreover, all items presented an adequate fit to the IRT Rasch model. Table 2 shows the descriptive statistics for the infit values of the most informative items for each age group.
The Socio-emotional domain presented one subscale (4 m-5 m) with an Alpha value below 0.70 and had two items replaced (SE17 by SE16, and SE21 by SE24). After modifications, the resulting Alpha and Omega reliabilities ranged from 0.76 to 0.94, and 0.81 to 0.97 (Table 2 and S2). Item SE50 had misfit for the subscales 30 m - 32 m, 33 m-35 m, 36 m-47 m, and 60 m-72 m and was replaced by item SE64. Items SE54 and SE57 were replaced by SE71 on the 60 m-65 m and 66 m-72 m subscales (Table S9). After alterations, the estimated psychometric parameters were considered acceptable.
All items from the Gross Motor subscales were adequate in terms of infit (Table 2 and S3). Reliabilities were adequate and ranged from 0.73 to 0.93 for Alpha and 0.82 to 0.94 for Omega estimates. The same pattern was observed for the items of the Fine Motor subscales in which reliabilities varied from 0.73 to 0.90 and 0.81 to 0.93 respectively, and all items fit adequately to the IRT model (Table S4).
One subscale (9 m - 11 m) from the Expressive Communication and Language domain had low Alpha reliability and had the CLE3 item replaced by CLE9. The 48 m - 59 m subscale had one item with misfit (CLE40 was replaced by CLE44). The 60 m - 65 m and 66 m - 72 m subscales had the CLE44 item replaced due to Alpha reliability and CLE40 due to misfit, both items were replaced by the CLE41 item (Tables 2 and S12). For the Receptive Communication and Language domains, all subscales had good to excellent reliabilities (ex variation: 0.70 to 0.97; © variation: 0.83 to 0.99). One item (CLR30) from the 48 m - 53 m, 60 m - 65 m, and 66 m - 72 m subscales had misfit and was replaced by item CLR31.
Three subscales - 18 m-23 m, 42 m-53 m, and 54 m-59 m - from the Adaptive Behavior domain presented misfitting items that were replaced by the next most informative items (CA16, CA21, and CA23 were replaced by CA15; CA32 by CA36; and CA59 was replaced by CA54). The 6 m-11 m subscale had the CA3 item replaced by C11 due to low reliability. After alterations, Alpha values ranged from 0.70 to 0.91 while Omega values ranged from 0.78 to 0.96 (Tables 2 and S14).
A sample of items from the final version of IDADI-B is available in the supplementary Table S16. Free access to all IDADI-B content is available at the following link: https://vol.vetoreditora.com.br/idadi breve.
Correlations and precision of developmental delay screening of IDADI-B in relation to IDADI.
A strong association was observed between the short and original versions of IDADI’s domains. Correlations varied from 0.75 to 0.89 with an average of 0.84 (Table 3). The precision for development delay screening was high with a sensitivity that varied from 0.90 to 0.97 and specificity that ranged from 0.87 to 0.93. The ROC area under the curve was highly significant with values that varied from 0.95 to 0.97. Table 4 shows the regression models considering developmental diagnosis, maternal education, family income, and sex of the child, simultaneously. The diagnostic predictor showed significant negative associations in all domains of IDADI-B with the largest effect sizes. Maternal education was positively associated with five domains (Cognitive, Receptive Communication and Receptive Language, Fine Motor, and Adaptive Behavior). Income was positively associated with Cognitive, Fine Motor, and Receptive Communication and Language scores. Girls had higher levels of development in all domains of IDADI-B except for the Gross Motor domain.
Discussion
This study used a rigorous quantitative approach when contrasted with the methods that have been used by other comparable assessments, to develop an empirically-based short version of the Dimensional Inventory for Child Development Assessment, IDADI-B. A large sample from different regions of Brazil was collected to investigate its psychometric properties and gather validity evidence with variables commonly associated with developmental outcomes. The results provided initial evidence of internal consistency and validity to support the interpretation and the free use of IDADI-B scores in research and clinical practice settings.
Psychometric analysis indicated that most items had a good fit for the IRT model. The use of IRT made it possible to evaluate the psychometric characteristics of IDADI-B that cannot be obtained using Classical Test Theory (CTT).24 It allowed us to evaluate the adequacy of an item to a onedimensional logistic model while modeling the probability of responses of the endorsed categories. As well, the item information curves made it possible to verify the precision of each item along the latent continuum under investigation.18 The fact that more than 95% of the 300 selected items had infit values within the acceptable range of 0.5-1.5 provided additional psychometric support, indicating that the items in the final version of IDADI-B fit well to the construct under investigation.
The finding that some items which demonstrated good fit in the original inventory but misfit in the short version has been observed in similar studies using the Ages and Stages Questionnaires (ASQ-3).20 A hypothesis raised was that an item may present problems in a specific age-group, but when considered in a wider ability range (multiple age-groups simultaneously) the residuals might be diluted. As the objective of this work was to validate individual subscales for each age group, only items with satisfactory adjustment were considered. It’s worth mentioning that CTT’s reliability estimates varied from adequate to excellent, also supporting the use of the scale using this measurement approach.23
The strong correlation with the original inventory and the high sensitivity and specificity for developmental delays highlight the good quality of the empirically-based item selection. Although not perfect, a comparison of IDADI-B scores with the original inventory point to a strong convergence using only a half to a third of IDADI’s items which ensures time efficiency. Such results are similar to the validation of Bayley’s developmental screening scale, which obtained high correlations with the original task-based scale.24
IDADI-B scores were also associated with several external criteria variables. Diagnosis was the variable most strongly associated with child development. Such results suggest the potential to identify children with developmental delays caused by neurodevelopmental disorders, one of the main objectives of the inventory. Although it is not characteristic for screeners to provide specific diagnoses, the results highlight the importance of monitoring the development of children, as the screening tools are highly sensitive to the identification of neurodevelopmental disorders such as Autism,25 disruptive behavior,26 and motor problems.27 Maternal education was associated with five IDADI-B domains and family income with three domains replicating the frequently observed effects of socioeconomic indicators on child development outcomes.16,28 Differences between children’s sex were also observed in other instruments such as the BSID and the ASQ,12,20 stressing the importance of considering the specificities between the sexes when dealing with development in early childhood.
Despite the potential benefit of using IDADI-B, some limitations should be addressed. The use of a new sample using the IDADI-B is needed to verify whether the results observed in the present study are replicable.23 The testretest analysis would constitute an important indicator of the degree of consistency of development estimates over a brief period of time. Validity evidence based on theoretically similar instruments such as the Denver Developmental Screening Test,29 ASQ-BR,20 or the Survey of Wellbeing of Young Children30 would also support the utility of the inventory. It is worth mentioning that, currently, IDADI is the only instrument specifically developed in Brazil. The ASQ-BR was validated only in the context of public day-cares from the state of Rio de Janeiro,20 while the BSID commercial version was only translated to Brazilian-Portuguese, and the norms of interpretation are based on an outdated American sample collected from years 2000 to 2002.12 IDADI, on the other hand, benefitted from a thorough content scrutiny of experienced professionals, semantic analysis of Brazilian mothers, and empirical validation data from all five regions of the country,14,15 thus making it an important tool for this population.
Taken together, these findings suggest that IDADI-B is a practical and psychometrically sound instrument for assessing the development of young children. The inventory is expected to improve the early detection rates of children with signs of developmental delays and facilitate referrals for interventions. This need is crucial since the earlier the referral, the greater the benefits, and chances are higher of mitigating the negative consequences of a disorder due to the greater brain plasticity in early childhood.28 Moreover, culturally appropriate measures of child development provide the means of evaluating the effectiveness of new interventions and the quality of health and educational programs.7,10 Screening instruments identify children who need more individualized attention, taking into account underor over-classification errors.2 It is known that transient qualitative aspects also characterize child development,1 and, therefore, pathologies need to be examined considering the environmentalrelational and neuroplastic aspects of this sensitive period of development.4,7,28 Hence, IDADI-B does not aim to be a long-term development predictor, on the contrary, it aims to provide a valid and reliable assessment of the child at the moment of screening. Professionals who use it should not draw any linear deterministic conclusions of future development, but use it as one indicator of the need for future intervention, paying special attention to the individualized development process of each child. It is expected that the identification of adversities in child development through IDADI-B, freely available at https://vol.vetoreditora.com.br/idadi breve, can benefit functional and structural aspects of child development opening the venue for intervention strategies implemented quickly to reduce risk factors and promote social protective factors for children and their families.
Acknowledgments
This research was supported by the University of Nebraska-Lincoln through its UNL/Brazil Collaboration Pilot Impact Grant; Fundação de Amparo a Pesquisa do Rio Grande do Sul (FAPERGS); and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES). Opinions expressed herein are those of the authors and do not reflect the position of the funders. IDADI-B is freely available at https://vol.vetoreditora.com.br/idadi breve. We thank the children and their families who participated in the study and generously contributed their time and effort.
References
- 1 Sameroff A. A unified theory of development: a dialectic integration of nature and nurture. Child Dev. 2010;81:6-22.
- 2 Bellman M, Byrne O, Sege R. Developmental assessment of children. BMJ. 2013;346:e8687.
- 3 Fernald LC, Kariger P, Engle P, Raikes A. Examining early child development in low-income countries: a toolkit for the assessment of children in the first five years of life. Washington, DC: The World Bank; 2009.
- 4 Fox SE, Levitt P, Nelson CA 3rd. How the timing and quality of early experiences influence the development of brain architecture. Child Dev. 2010;81:28-40.
- 5 Feldman R. Sensitive periods in human social development: New insights from research on oxytocin, synchrony, and high-risk parenting. Dev Psychopathol. 2015;27:369-95.
- 6 Almas AN, Degnan KA, Nelson CA, Zeanah CH, Fox NA. IQ at age 12 following a history of institutional care: Findings from the Bucharest Early Intervention Project. Dev Psychol. 2016;52:1858-66.
- 7 Doyle O, Harmon CP, Heckman JJ, Tremblay RE. Investing in early human development: timing and economic efficiency. Econ Hum Biol. 2009;7:1-6.
- 8 Knudsen EI. Sensitive periods in the development of the brain and behavior. J Cogn Neurosci. 2004;16:1412-25.
- 9 Rosenberg SA, Zhang D, Robinson CC. Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics. 2008;121:e1503-9.
- 10 Gleason MM, Zeanah CH, Dickstein S. Recognizing young children in need of mental health assessment: Development and preliminary validity of the early childhood screening assessment. Infant Ment Health J. 2010;31:335-57.
- 11 da Becker SM, Bandeira C de M, Ghilardi RB, Hutz CS, Piccinini CA. Psicologia do Desenvolvimento Infantil: Publicações nacionais na primeira década do século XXI. Psico. 2013;44:372-81.
- 12 Bayley N. Escala Bayley de desenvolvimento do bebê e da criança pequena. 3rd ed. São Paulo: Pearson Clinical Brasil; 2018.
- 13 Silva MA, de Mendonça Filho EJ, Bandeira DR. Inventário Dimensional de Avaliação do Desenvolvimento Infantil. São Paulo: Vetor; 2020.
- 14 Silva MA, de Mendonça Filho EJ, Bandeira DR. Development of the Dimensional Inventory of Child Development Assessment (IDADI). Psico-USF. 2019;24:11-26.
- 15 de Mendonça Filho EJ. Inventário Dimensional de Avaliação do Desenvolvimento Infantil: normas, estudo de comparação e versão breve [Doctoral dissertation]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2020.
-
16 Cauduro GN, de Mendonça Filho EJ, Pandolfo N, Bandeira DR. Direct and indirect effects of socioeconomic status on child development: is developmental parenting a relevant mediator? Early Child Dev Care. 2019, http://dx.doi.org/10.1080/03004430.2019.1673384
» http://dx.doi.org/10.1080/03004430.2019.1673384 - 17 Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978;43:561-73.
- 18 de Ayala RJ. The theory and practice of item response theory. New York, NY: The Guilford Press; 2009.
- 19 Glascoe FP. Parents’ Evaluation of Developmental Status: An Evidence-Based Approach to Developmental Behavioral Screening and Surveillance. Nashville, TN: Ellsworth & Vandermeer Press; 2007.
- 20 Filgueiras A, Pires P, Maissonette S, Landeira-Fernandez J. Psychometric properties of the Brazilian-adapted version of the Ages and Stages Questionnaire in public child daycare centers. Early Hum Dev. 2013;89:561-76.
- 21 Linacre JM. What do infit and outfit, mean-square and standardized mean? Rasch Meas Trans. 2002;16:878.
- 22 Dunn TJ, Baguley T, Brunsden V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014;105:399-412.
- 23 American Educational Research Association. American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington: American Educational Research Association; 2014.
- 24 Guedes DZ, Primi R, Kopelman BI. BINS validation - Bayley neurodevelopmental screener in Brazilian preterm children under risk conditions. Infant Behav Dev. 2011;34:126-35.
- 25 Wiggins LD, Piazza V, Robins DL. Comparison of a broad-based screen versus disorder-specific screen in detecting young children with an autism spectrum disorder. Autism. 2014;18:76-84.
- 26 Szczepaniak D, McHenry MS, Nutakki K, Bauer NS, Downs SM. The prevalence of at-risk development in children 30 to 60 months old presenting with disruptive behaviors. Clin Pediatr (Phila). 2013;52:942-9.
- 27 Vanvuchelen M, Van Schuerbeeck L, Braeken MA. Screening accuracy of the parent-completed Ages and Stages Questionnaires - second edition as a broadband screener for motor problems in preschoolers with autism spectrum disorders. Autism. 2017;21:29-36.
- 28 Shonkoff JP, Garner AS, Committee on Psychosocial Aspects of Child and Family Health; Committee on Early Childhood, Adoption, and Dependent Care; Section on Developmental and Behavioral Pediatrics. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129:e232-46.
- 29 Frankenburg WK, Dodds J, Archer P, Shapiro H, Brensnick B. Denver II: Teste de triagem do desenvolvimento. São Paulo, SP: Hogrefe; 2018.
- 30 Moreira RS, Magalhães L de C, Siqueira CM, Alves CR. Crosscultural adaptation of the child development surveillance instrument ‘‘Survey of Wellbeing of Young Children (SWYC)’’ in the Brazilian context. J Hum Growth Dev. 2019;29:268-78.
Publication Dates
-
Publication in this collection
06 Dec 2021 -
Date of issue
2021
History
-
Received
24 Sept 2020 -
Accepted
20 Jan 2021 -
Published
24 Feb 2021