Comparison between self-reported and accelerometer-derived measurements for classifying children and adolescents as physically active in Chile

Comparación entre medidas autoreportadas y provenientes de acelerómetro para clasificar a los niños, niñas y adolescentes como físicamente activos en Chile

Comparação de autorrelato e medidas derivadas de acelerômetro para classificar a atividade física em crianças e adolescentes chilenos

Abstracts

The aim of this study was to compare self-reported with two accelerometer-derived methods to classify Chilean children and adolescents as physically active. In total, 247 students wore an accelerometer on their hips during 7 consecutive days to classify them as physically active based on (1) daily accumulation of ≥ 60 minutes of moderate-to-vigorous physical activity (MVPA) on each of the seven days, and (2) average MVPA ≥ 60 minutes/day. Also, participants were classified as physically active if they reported being active for at least 60 minutes in all seven days. When using the accelerometer data, 0.8% were active in all seven days, while 10.5% recorded ≥ 60 minutes MVPA per day on average. Based on self-report, 7.2% were physically active. The agreement between self-reported and accelerometer estimations were poor. Important differences were observed between the self-reported and device-derived methods for classifying children and adolescents as physically active. When comparing them, some considerations should be taken. The findings suggest that these methods are not interchangeable. Therefore, if possible, they should be used as complementary measurements.

Keywords: Exercise; Motor Activity; Movement; Child; Adolescent

El objetivo fue comparar el autoreporte con dos métodos provenientes de acelerómetría para clasificar a niños, niñas y adolescentes chilenos como físicamente activos. Doscientos cuarenta y siete estudiantes llevaron un acelerómetro en la cintura durante siete días consecutivos y fueron clasificados como físicamente activos, basados en: (1) acumulación diaria de ≥ 60 minutos de actividad física de moderada a vigorosa (AFMV) en cada uno de los siete días, y (2) promedio de AFMV ≥ 60 minutos/día. Asimismo, los participantes fueron clasificados como físicamente activos si reportaron ser activos por al menos 60 minutos en cada uno de los siete días. Al usar los datos del acelerómetro, 0,8% fueron activos los siete días, mientras que un 10,5% registró ≥ 60 minutos AFMV por día promedio. Un 7,2% fue físicamente activos según el autoreporte. El acuerdo entre las estimaciones autoreportadas y el acelerómetro fue pobre. Se observaron importantes diferencias entre los autoreportes y los métodos derivados del dispositivo para clasificar niños, niñas y adolescentes como físicamente activos. Cuando se comparen datos derivados de los acelerómetros y autoreportes esto se debería considerar. Los resultados sugieren que estos métodos no son intercambiables. Por ello, en la medida de lo posible, se deberían usar como medidas complementarias.

Palabras-clave:  Ejercicio Físico; Actividad Motora; Movimiento; Niño; Adolescente

O objetivo foi comparar o autorrelato com dois métodos derivados de acelerômetro para classificar crianças e adolescentes chilenos como fisicamente ativas ou inativas. Um total de 247 alunos usaram um acelerômetro no quadril durante sete dias consecutivos e foram classificados como fisicamente ativos com base em: (1) acúmulo diário de ≥ 60 minutos de atividade física de moderada a vigorosa intensidade (AFMV) em cada um dos sete dias e (2) AFMV média por dia de ≥ 60 minutos. Além disso, os participantes foram classificados como fisicamente ativos com base no autorrelato de ser ativo por pelo menos 60 minutos em todos os sete dias. Com o uso dos dados de acelerômetro, 0,8% foram classificados como ativos em todos os sete dias, enquanto 10,5% registravam ≥ 60 minutos de AFMV por dia em média e 7,2% eram fisicamente ativos com base no autorrelato. A concordância foi baixa entre o autorrelato e as estimativas por acelerômetro. Foram observadas diferenças importantes entre o autorrelato e os métodos baseados em dispositivos para classificar as crianças e adolescentes como fisicamente ativos. Algumas considerações são relevantes ao comparar estimativas baseadas em acelerômetro e o autorrelato de atividade física. Os achados sugerem que esses métodos não são intercambiáveis. Portanto, quando possível, devem ser usados como medidas complementares.

Palavras-chave:  Exercício Físico; Atividade Motora; Movimento; Criança; Adolescente

Introduction

Physical activity recommendations suggest that children and adolescents should accumulate at least 60 minutes of moderate to vigorous physical activity (MVPA) per day to achieve optimal health benefits ¹. Studies have shown that physical activity is associated with multiple beneficial outomes in cardiorespiratory and muscular fitness, bone health, cognition, mental health and academic outcomes ^2,3,4. However, physical inactivity is highly prevalent in children and adolescents worldwide ⁵, and particularly in Chile, only 2 out of 5 adolescents are considered physically active ⁶.

Although physical activity recommendations have been initially analyzed with self-reports, recent studies have explored them with the use of accelerometers ^3,7. Studies conducted with accelerometry have used different criteria to classify children and adolescents as physically active ^7,8,9. For example, doing 60 minutes or more of MVPA per day on average, or if they accumulate 60 minutes of MVPA each day ^3,7. Also, there are several wearing time protocols for the accelerometers to validate participation ³, along with different accelerometer brands and placements (e.g., hip or wrist) ¹⁰. These different methodologies for data collection and processing yield different estimates that may limit their comparability, especially in self-reported methods ^8,9.

There have been efforts to compare physical activity guidelines compliance from device-derived data ^3,5,7. Still, there is limited understanding of exchangeability between accelerometer-derived and self-reported estimates, particularly across different age groups, sex or countries. By understanding the comparability between methods, researchers may still implement studies using only self-report methods in settings where resources are scarce. It may also be possible to pool and harmonize data from different sources to increase the representation of countries in global analyses and to help interpret dose-response relationships between physical activity and different health outcomes. Therefore, the aim of this study was to compare and assess the agreement between self-reported and two accelerometer-derived methods to classify Chilean children and adolescents as physically active.

Methods

Students aged between 9 and 15 were recruited from randomly selected classes from schools in two Chilean cities (Temuco and Carahue). The school selection process (n = 16) was stratified considering location (city) and socioeconomic status of the school. After each school was selected, two classes were randomly selected. A total of 565 participants was estimated for the original study, considering the proportion of inactive children and adolescents in Chile ¹¹, a 5% significance, an absolute precision of 5%, the effect size of 1.3 based on the cluster design ¹², and a response rate of 80%. Parents signed an informed consent form and all participants gave their signed consent. The study was approved by the Ethics Scientific Committee at the Universidad de La Frontera (Chile).

The participants wore an accelerometer (ActiGraph GT3X+; https://actigraphcorp.com/) on the right hip for seven consecutive days. Participants that wore the accelerometer at least 10 hours/day in seven days were included in the analysis. Evenson cut-points were used to identify time spent in MVPA with the accelerometer (≥ 574 counts per 15 seconds from the vertical axis) ⁷.

Two criteria derived from accelerometry were used to classify the participants as physically active:

(1) Daily accumulation of MVPA (i.e., DAILY): at least 60 minutes of MVPA on each of the seven days; and

(2) Average accumulation of MVPA per day (i.e., AVERAGE): at least 60 minutes of MVPA per day on average.

Participants answered: “During the past seven days, on how many days were you physically active for a total of at least 60 minutes?” and were classified as physically active if they reported being active “in all seven days,” as recommended by the World Health Organization (WHO) ^13,14.

Age was categorized as 9-11 years old and 12-15 years old. Nutritional status categories were obtained from body mass index according to in-field measurements of weight and height, using the WHO criteria ¹⁵.

Sociodemographic and physical activity-related variables were compared using t-tests, chi-squared and proportion tests, and kappa statistic. The following interpretation for the kappa results was used: 0.01-0.20 as none to poor; 0.21-0.40 as fair; 0.41-0.60 as moderate; 0.61-0.80 as substantial; and 0.81-1.00 as almost perfect agreement. Receiver operating characteristic (ROC) analysis was used to assess classification accuracy for both accelerometer criteria and the self-report. All results were stratified by gender. The statistical analysis was performed with Stata 15.0 (https://www.stata.com) and the significance level was set at p < 0.05.

Results

In total, 837 participants completed the study, but only 237 (30.6%) wore the accelerometer during seven consecutive days (12.2 ± 1.49 years; 43.5% male) (Table 1). Based on self-report, only 7.2% were physically active. When using accelerometer data, only 0.8% were physically active in all seven days, while 10.5% met the physical activity recommendations with the AVERAGE method (p < 0.0001). Large differences were observed between self-reported physical activity compliance and the DAILY method (7.7% vs. 0.8%; p < 0.001). No differences were observed when comparing self-report with the AVERAGE method (7.7% vs. 10.5%; p = 0.196).

Table 1
Participants’ characteristics.

Characteristics	Total (n = 237)	Boys (n = 103)	Girls (n = 134)	p-value
Age in years [mean (SD)]	12.2 (1.49)	12.2 (1.32)	12.1 (1.62)	0.535
Nutritional status [%]
Normal	54.5	55.9	53.4	0.907
Overweight	37.9	36.3	39.1
Obesity	7.6	7.8	7.5
Self-reported physical activity
Meeting physical activity guidelines [%]	7.2	9.7	5.2	0.185
9-11 years old	10.0	16.0	5.7	0.190
12-15 years old	6.2	7.7	5.1	0.470
Physically active days [mean, SD]	3.1 (1.91)	3.3 (1.96)	2.9 (1.84)	0.141
9-11 years old	2.8 (2.06)	2.9 (2.32)	2.8 (1.91)	0.901
12-15 years old	3.2 (1.84)	3.4 (1.83)	3.0 (1.83)	0.108
Device-measured physical activity
Meeting physical activity with DAILY method * [%]	0.8	1.9	0.0	0.105
9-11 years old	1.7	4.0	0.0	0.233
12-15 years old	0.6	1.3	0.0	0.259
Number of active days (≥ 60 minutes MVPA) [mean, SD]	0.9 (1.58)	1.1 (1.58)	0.8 (1.29)	0.140
9-11 years old	2.2 (1.83)	2.7 (2.01)	1.8 (1.61)	0.054
12-15 years old	0.5 (1.23)	0.6 (1.53)	0.4 (0.93)	0.445
Meeting physical activity with AVERAGE method ** [%]	10.5	15.5	6.7	0.028
9-11 years old	30.0	40.0	22.9	0.153
12-15 years old	4.0	7.7	1.0	0.024
Mean MVPA minutes per day [mean, SD]	25.4 (24.16)	28.1 (28.74)	23.3 (19.79)	0.129
9-11 years old	49.1 (23.04)	58.0 (26.72)	42.8 (17.87)	0.011
12-15 years old	17.4 (18.65)	18.6 (22.14)	16.4 (15.40)	0.450

• MVPA: moderate-to-vigorous physical activity; SD: standard deviation.
• * DAILY: physically active defined as accumulating at least 60 minutes of MVPA on every day;
• ** AVERAGE: physically active defined as the average accumulation of at least 60 minutes of MVPA per day.

Overall, sensitivity was lower than specificity when comparing both the DAILY and AVERAGE methods with self-report (Table 2). ROC areas were higher for the AVERAGE method than the DAILY method. When comparing the ROC area and predictive properties, the DAILY and AVERAGE methods performed better in those aged from 9 to 11 years old, when compared with those aged between 12 to 15 years old. The overall agreement between self-report and both device-derived criteria was poor, but fair for the younger group.

Table 2
Classification properties of the accelerometer-derived estimations compared with the self-reported compliance of physical activity recommendations using a single question.

	DAILY * vs. self-report			AVERAGE ** vs. self-report
	Total	Boys	Girls	Total	Boys	Girls
Total sample
Sensitivity (%)	5.9	10.0	NA	29.4	40.0	14.3
Specificity (%)	99.5	98.9	NA	90.9	87.1	93.7
ROC area	0.53	0.54	NA	0.60	0.64	0.54
Positive predictive value (%)	50.0	50.0	NA	20.0	25.0	11.1
Negative predictive value (%)	93.2	91.1	NA	94.3	93.1	95.2
Kappa	0.09 (p = 0.009)	0.14 (p = 0.026)	NA	0.17 (p = 0.004)	0.21 (p = 0.012)	0.07 (p = 0.206)
9-11 years old
Sensitivity (%)	16.7	25.0	NA	66.7	75.0	50.0
Specificity (%)	100	100	NA	74.1	66.7	78.8
ROC area	0.58	0.63	NA	0.70	0.71	0.64
Positive predictive value (%)	100.0	100.0	NA	22.2	30.0	12.5
Negative predictive value (%)	91.5	87.5	NA	95.2	93.3	96.3
Kappa	0.26 (p = 0.001)	0.36 (p = 0.010)	NA	0.22 (p = 0.019)	0.26 (p = 0.060)	0.12 (p = 0.173)
12-15 years old
Sensitivity (%)	0.0	0.0	NA	9.1	16.7	0.0
Specificity (%)	99.4	98.6	NA	96.4	93.1	98.9
ROC area	0.50	0.49	NA	0.53	0.55	0.49
Positive predictive value (%)	0.0	0.0	NA	14.3	16.7	0.0
Negative predictive value (%)	93.8	92.2	NA	94.1	93.1	94.9
Kappa	-0.01 (p = 0.602)	-0.02 (p = 0.614)	NA	0.07 (p = 0.183)	0.10 (p = 0.195)	-0.02 (p = 0.592)

• MVPA: moderate-to-vigorous physical activity; NA: not applicable as no physically active girls were observed when using this criterion; ROC: receiving operating characteristic.
• * DAILY: physically active defined as accumulating at least 60 minutes of MVPA on every day;
• ** AVERAGE: physically active defined as the average accumulation of at least 60 minutes of MVPA per day.

Discussion

Overall, 7.2% of the participants were physically active based on self-report. When using accelerometer data, 0.8% were active in all seven days, while 10.5% recorded ≥ 60 minutes MVPA per day on average. The current study showed that physical activity compliance rates derived from self-reported and device-derived methods might not be interchangeable in children and adolescents.

Although the overall prevalence - derived from the self-report and the AVERAGE methods - showed no significant differences (7.2% vs. 10.5%; p = 0.196), our findings suggest that these are not interchangeable, based on the accuracy classification. In line with other studies ^8,9, when comparing compliance rates in children aged from 12 to 15 years old, the predictive properties were extremely low. In most cases, any similar percentage in the overall estimates between the self-report and device-derived methods is highly likely to be a coincidence. For example, children aged from 9 to 11 years old, based on the AVERAGE method, were three times more active than those classified with a single question (30% vs. 10%; p = 0.006). Therefore, to avoid misinterpretations when comparing studies, it is essential to identify how the physical activity compliance rates were calculated from both self-reports and accelerometers.

Both the DAILY and AVERAGE methods yielded different percentages of physical activity compliance. This aspect has relevant implications for reporting procedures on accelerometer data cleaning and extraction to facilitate data comparability between studies ¹⁶. Also, researchers must communicate not only the method used to estimate (for example, the total time spent in MVPA), but also the accelerometer wear protocol (minimum hours and days), epochs, type of data processed (e.g., raw or counts), predictive models to calculate physical activity intensity (e.g., equations), among other decisions. The accelerometer location plays a crucial role too, as studies have reported differences in physical activity estimates between hip- and wrist-worn devices ^16,17. For example, a study conducted in children aged 9-12 found that wrist-worn accelerometers detected more activity than the hip-worn devices ¹⁷.

Future studies should explore how these methodological decisions may have implications when predicting associations between physical activity guidelines compliance and health outcomes ³. Some data extraction methods may hinder some relationships with key outcomes, if these differences in the estimates are not considered in data processing ¹⁸. Still, we suggest that both self-report and accelerometer approaches should be used as complementary tools, as they have shown consistent positive associations with physical, mental and social health ^1,3,7. On the other hand, few moderate to high quality studies have explored the dose-response relationship between physical activity and health outcomes in adults ¹⁹. Also, physical activity is highly variable within a week and a year ^20,21,22. In this case, the exclusive use of questionnaire in the current form may limit the exploration of these associations. Some children may experience benefits from activity even if they accumulate physical activities in some days of the week, as recently recommended in the new WHO guidelines on physical activity and sedentary behavior ²³. Therefore, future research should assess how average physical activity may benefit children and adolescents, not only the achievement of 60 minutes of physical activity per day.

Accelerometers provide a unique opportunity not only to classify physically active individuals, but also to capture other physical activity features. Some essential aspects to consider are movement pattern variability within and between days and how different movement behaviors interact with each other (i.e., sleeping, sedentary behavior, and physical activity). There are increased chances to better understand movement, but at the same time, the availability of different devices and protocols have limited exchangeability. The use of raw data offers a unique opportunity for promoting collective efforts, as it is possible to harmonize data independently of the device brand ¹⁰. Therefore, continuous international collaborations are vital to develop reporting standards to facilitate data harmonization and comparisons ^7,24.

To our knowledge, this is the first study that has explored the classification agreement between the questions and the criterion used by the WHO and accelerometers to classify Chilean children and adolescents as physically active. We had a relatively large sample, but only a third provided enough data for this study. To be strict with the recommendations, we only included participants with a minimum of seven days of valid accelerometer data, resulting in low compliance. In our sample, children with less than seven valid days were more likely to be boys (56.8% vs. 43.2%; p = 0.006), obese (18.2% vs. 7.6%; p = 0.005), younger (11.5 vs. 12.1 years; p < 0.001), and accumulated more average MVPA per day (34.1 vs. 25.4 minutes/day; p = 0.001). This information may be relevant for future study design and analysis.

Conclusion

Important differences and poor to fair agreement were observed between the self-report and device-derived methods for classifying children and adolescents as physically active. The AVERAGE method provided similar overall estimates of physical activity guidelines compliance to self-report, but the classification accuracy was poor. Researchers should consider these differences when comparing or pooling data as some estimates may differ considerably. We suggest that both self-report and accelerometer approaches should be used, if possible, as complementary tools as their estimates are not comparable or exchangeable; but provide valuable insights about movement behaviors in children and adolescents.

Acknowledgments

We would like to acknowledge and pay our respects to the traditional owners and custodians of Wallmapu, where we conducted the study. The authors would like to thank the participants, caregivers, teachers, and school principals for collaborating with this study. This study was funded by CONICYT FONDECYT 11160720 and the Ministry of Education of Chile and Universidad de La Frontera (UFRO) throughout the Institutional Improvement Plan UNETE, Universidad es Territorio, FRO 1301 (UNT14-008). Special thanks to UFRO activate research team for their assistance.

References

• ¹ Janssen I, Leblanc AG. Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. Int J Behav Nutr Phys Act 2010; 7:40.
• ² Tremblay MS, Carson V, Chaput JP, Gorber SC, Dinh T, Duggan M, et al. Canadian 24-hour movement guidelines for children and youth: an integration of physical activity, sedentary behaviour, and sleep. Appl Physiol Nutr Metab 2016; 41(6 Suppl 3):S311-27.
• ³ Poitras VJ, Gray CE, Borghese MM, Carson V, Chaput JP, Janssen I, et al. Systematic review of the relationships between objectively measured physical activity and health indicators in school-aged children and youth. Appl Physiol Nutr Metab 2016; 41(6 Suppl 3):S197-239.
• ⁴ Department of Health. Australian 24-hour movement guidelines for children and young people (5 to 17 years): an integration of physical activity, sedentary behaviour, and sleep. Canberra: Australian Government; 2019.
• ⁵ Aubert S, Barnes JD, Abdeta C, Nader PA, Adeniyi AF, Aguilar-Farias N, et al. Global matrix 3.0 physical activity report card grades for children and youth: results and analysis from 49 countries. J Phys Act Health 2018; 15(S2):S251-73.
• ⁶ Aguilar-Farias N, Miranda-Marquez S, Martino-Fuentealba P, Sadarangani KP, Chandia-Poblete D, Mella-Garcia C, et al. 2018 Chilean physical activity report card for children and adolescents: full report and international comparisons. J Phys Act Health 2020; 17:807-15.
• ⁷ Steene-Johannessen J, Hansen BH, Dalene KE, Kolle E, Northstone K, Moller NC, et al. Variations in accelerometry measured physical activity and sedentary time across Europe - harmonized analyses of 47,497 children and adolescents. Int J Behav Nutr Phys Act 2020; 17:38.
• ⁸ Mooses K, Mäestu J, Riso E-M, Hannus A, Mooses M, Kaasik P, et al. Different methods yielded two-fold difference in compliance with physical activity guidelines on school days. PLoS One 2016; 11:e0152323.
• ⁹ Price L, Wyatt K, Lloyd J, Abraham C, Creanor S, Dean S, et al. Are we overestimating physical activity prevalence in children? J Phys Act Health 2018; 15:941-5.
• ¹⁰ Migueles JH, Rowlands AV, Huber F, Sabia S, van Hees VT. GGIR: a research community-driven open source R package for generating physical activity and sleep outcomes from multi-day raw accelerometer data. J Meas Phys Behav 2019; 2:188-96.
• ¹¹ Godard C, Román M, Rodríguez M, Leyton B, Salazar G. Variabilidad de la actividad física en niños chilenos de 4 a 10 años: estudio por acelerometría. Arch Argent Pediatr 2012; 110:388-93.
• ¹² Williamson DA, Champagne CM, Harsha DW, Han H, Martin CK, Newton Jr. RL, et al. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: a randomized trial. Obesity (Silver Spring) 2012; 20:1653-61.
• ¹³ Aguilar-Farias N, Martino-Fuentealba P, Carcamo-Oyarzun J, Cortinez-O'Ryan A, Cristi-Montero C, Von Oetinger A, et al. A regional vision of physical activity, sedentary behaviour and physical education in adolescents from Latin America and the Caribbean: results from 26 countries. Int J Epidemiol 2018; 47:976-86.
• ¹⁴ World Health Organization. Global school-based student health survey (GSHS). https://www.who.int/ncds/surveillance/gshs/methodology/en (accessed on 05/May/2020).
  » https://www.who.int/ncds/surveillance/gshs/methodology/en
• ¹⁵ de Onis M, Onyango AW, Borghi E, Siyam A, Nishida C, Siekmann J. Development of a WHO growth reference for school-aged children and adolescents. Bull World Health Organ 2007; 85:660-7.
• ¹⁶ Noonan RJ, Boddy LM, Kim Y, Knowles ZR, Fairclough SJ. Comparison of children's free-living physical activity derived from wrist and hip raw accelerations during the segmented week. J Sports Sci 2017; 35:2067-72.
• ¹⁷ McLellan G, Arthur R, Buchan DS. Wear compliance, sedentary behaviour and activity in free-living children from hip-and wrist-mounted ActiGraph GT3X+ accelerometers. J Sports Sci 2018; 36:2424-30.
• ¹⁸ Martinez-Gomez D, Gomez-Martinez S, Ruiz JR, Diaz LE, Ortega FB, Widhalm K, et al. Objectively-measured and self-reported physical activity and fitness in relation to inflammatory markers in European adolescents: the HELENA Study. Atherosclerosis 2012; 221:260-7.
• ¹⁹ World Health Organization. WHO guidelines on physical activity and sedentary behaviour: web annex evidence profiles. Geneva: World Health Organization; 2020.
• ²⁰ Pereira S, Gomes TN, Borges A, Santos D, Souza M, Santos FK, et al. Variability and stability in daily moderate-to-vigorous physical activity among 10 year old children. Int J Environ Res Public Health 2015; 12:9248-63.
• ²¹ Nilsson A, Anderssen SA, Andersen LB, Froberg K, Riddoch C, Sardinha LB, et al. Between- and within-day variability in physical activity and inactivity in 9- and 15-year-old European children. Scand J Med Sci Sports 2009; 19:10-8.
• ²² Carson V, Spence JC. Seasonal variation in physical activity among children and adolescents: a review. Pediatr Exerc Sci 2010; 22:81-92.
• ²³ World Health Organization. WHO guidelines on physical activity and sedentary behaviour. Geneva: World Health Organization; 2020.
• ²⁴ Cain KL, Sallis JF, Conway TL, Van Dyck D, Calhoon L. Using accelerometers in youth physical activity studies: a review of methods. J Phys Act Health 2013; 10:437-50.

Publication Dates

• Publication in this collection
  22 Feb 2021
• Date of issue
  2021

History

• Received
  12 Aug 2020
• Reviewed
  01 Dec 2020
• Accepted
  17 Dec 2020