Abstract
Objective: to evaluate the accuracy of an antibody point-of-care lateral flow immunoassay (LFI -Wondfo Biotech Co., Guangzhou, China) in a pediatric population.
Methods: children and adolescents (2 months to 18 years) with signs and symptoms suggestive of acute SARS-CoV-2 infection were prospectively investigated with nasopharyngeal RT-PCR and LFI at the emergency room. RT-PCR was performed at baseline, and LFI at the same time or scheduled for those with less than 7 days of the clinical picture. Overall accuracy, sensitivity and specificity were assessed, as well as according to the onset of symptoms (7-13 or ≥14 days) at the time of the LFI test.
Results: In 175 children included, RT-PCR and LFI were positive in 51 (29.14%) and 36 (20.57%), respectively. The overall sensitivity, specificity, positive and negative predictive value was 70.6% (95%CI 56.2-82.5), 96.8% (95%CI 91.9-99.1), 90.0% (95%CI 77.2-96.0), and 88.9% (95%CI 83.9-92.5), respectively. At 7-13 and ≥14 days after the onset of symptoms, sensitivity was 60.0% (95%CI 26.2-87.8) and 73.2% (95%CI 57.1-85.8) and specificity was 97.9% (95%CI 88.7-99.9) and 96.1% (95%CI 89.0-99.2), respectively.
Conclusion: Despite its high specificity, in the present study the sensitivity of LFI in children was lower (around 70%) than most reports in adults. Although a positive result is informative, a negative LFI test cannot rule out COVID-19 in children.
KEYWORDS Children; SARS-CoV-2; COVID-19; Lateral flow immunoassay; Point-of-Care Testing
Introduction
The COVID-19 pandemic is the largest world health crisis of the century.1,2 The limitations of the effectiveness of the containment measures and the great impact in several aspects of society forced world leaders to make difficult decisions.3,4,5 The strategies to mitigate the pandemic include social distancing, use of face masks, hand hygiene, vaccines and the widespread use of accurate testing.6,7
Despite the large number of studies published since the beginning of the pandemic, the knowledge about SARS-CoV-2 infection tests accuracy is still evolving. As critical cases of COVID-19 were far more common in older adults and patients with comorbidities, most studies assessing the accuracy of diagnostic tests were performed in this population.8,9,10,11
Although children and adolescents with COVID-19 usually have a mild disease, severe cases occur,12 as is the case of the rare COVID-19-related multisystem inflammatory syndrome (MIS-C). This syndrome resembles Kawasaki-disease and can lead to multiorgan failure and even death.13,14,15 Also, children may play a role in SARS-CoV-2 spread, although the viral dynamics in the pediatric population are not fully understood.16,17,18,19 All this novel information justifies more precise definitions of the best accuracy of viral diagnostic tools, as an important public health preventive measure among the pediatric population.
The aim of this study was to assess the accuracy of an antibody LFI test in children and adolescents compared to RT-PCR in the acute care setting. The authors have also compared the test accuracy between infants and older children.
Materials and methods
This is a prospective multicenter observational study with data collected in two hospitals, from May to November 2020. Pediatric patients (>2 months and 18 years) admitted at emergency rooms (ERs) or visiting outpatient clinics with signs or symptoms suggestive of COVID-19 (cough, fever, or sore throat) within 14 days of onset of symptoms were eligible. The exclusion criteria was the failure to collect samples for SARS-CoV-2 RT-PCR. Legal caregivers provided signed consent and authorization for their child’s participation. The study was carried out according to the Declaration of Helsinki and Good Clinical Practice Guidelines, after approval by the study’s Institutional Review Board (IRB No. 30749720.4.1001.5330).
The identification of SARS-CoV-2 through RT-PCR assays was performed in parallel for all participants. Nasopharyngeal and oropharyngeal swab sample collections were allocated in the same transport media with saline solution and RNAlater®, and RNA Stabilization Solution (Catalog number AM7021). RNA extraction was performed using MagMax™ Viral/Pathogenic Nucleic Acid Isolation (Applied Biosystems) in KingFisher Duo Prime System platform, (ThermoFisher, USA). The total reaction volume was 10 µL, with 5 µL of Path™ 1-Step RT-qPCR Master Mix, CG (Catalog Numbers A15299, AppliedBiosystems), TaqMan™ 2019-nCoVAssay Kit v1 (Catalog Number A47532), and 5 µL of RNA. Reaction control had 5 µL (200 copies/ µL) of TaqMan™ 2019-nCoV Control Kit v1 (Catalog Number A47533). RT-PCR assay was performed on QuantStudio 5 (ThermoFisher Scientific, USA) and results were analyzed with QuantStudio™ Design & Analysis Software v1.5.1.
The SARS-CoV-2 Antibody Test (Wondfo Biotech Co., Guangzhou, China), was used to assess the presence of antibodies in two drops of blood from a finger-prick sample. This lateral flow assay detects immunoglobulins (Ig) G and M isotypes specific to the spike protein of SARS-CoV-2 receptor binding domain. The LFI assay reagent consists of colloidal gold particles coated with SARS-CoV-2 antigen-dye conjugate of receptor-binding domain (RBD; personal communication from the manufacturer). A positive result is characterized by the presence of human IgM and/or IgG SARS-CoV-2 specific antibodies in the blood sample. SARS-CoV-2 antibodies bind to antigen-dye conjugate and to colloidal gold complexes, defining a line test (T) in the kit’s window, presenting as a dark-colored band. Instead, samples without SARS-CoV-2 specific antibodies will not display this dark-colored band, indicating a negative result. All valid tests presented a positive control band (C). If this control band was not visible, the test was considered invalid and the participant retested. If there was doubt about the result another researcher evaluated the result or the test was repeated.
RT-PCR sample collection was performed at baseline and LFI at the same time or scheduled for later in those with less than 7 days of clinical onset. All tests were performed through a standardized protocol by the trained study team. The laboratory team was blind to the LFI test results until the final analysis. However, when LFI was not performed at the inclusion, the research team had access to the RT-PCR test results.
Mean and standard deviation or median and interquartile range (IQR) were calculated for continuous variables and percentages for the categorical variables to describe the characteristics of the subject. Diagnostic accuracy was calculated using the epiR package (version 1.0-15). Sensitivity was characterized as the number of positive LFI test results, divided by the number of positive RT-PCR. Specificity was calculated as the number of negative LFI results divided by the total number of negative RT-PCR. Positive predictive value (PPV) was characterized as the number of positive LFI and RT-PCR test results (true positives) divided by the number of total LFI positives. Negative predictive value (NPV) was characterized as the number of negative LFI and RT-PCR test results (true negatives) divided by the number of total LFI negatives. The 95% confidence intervals for all estimates were calculated using the exact method.
To explore the accuracy according to the onset of symptoms, the authors calculated days of symptoms onset at the time of LFI testing, and assessed sensitivity and specificity at 7-13 days, and ≥14 days. The authors performed a sensitivity sub-analysis to compare the accuracy of LFI between infants (<2 years old) and older children (≥2 years old). Patients without both LFI and RT-PCR results were excluded from the analysis. All analyses were performed in Rversion 3.6.3
Results
383 subjects were screened for the study, and 208 excluded: 16 for not meeting inclusion criteria, 13 for not consenting, and 179 did not perform LFI, as shown in Figure 1. A total of 175 participants were included in the final analysis, and SARS-CoV-2 infection was confirmed by RT-PCR in 51 (29.14%), while 36 (20.57%) were positive on the LFI. The authors observed 4 (2.28%) cases of negative RT-PCR with positive LFI. The comparison between included patients and those who did not perform LFI is shown in the Supplementary table 1.
The most prevalent symptoms reported were cough (77.71%), coryza (72.57%), fever (66.29%), headache (59.43%), and stuffy nose (57.71%). Demographic and clinic details are presented in Table 1. The median (IQR) time from the onset of symptoms to RT-PCR assay was 3 (2-7) days. Twenty-eight patients performed RT-PCR after 7 days of onset of symptoms (23 with negative results). The median (IQR) time from the onset of symptoms to LFI testing was 16 (10-18) days.
Overall sensitivity and specificity (≥7 days of symptom onset) was 70.6% (95%CI 56.2-82.5) and 96.8% (95%CI 91.9-99.1), respectively (Table 2). Positive predictive value was 90% (95% CI 77.0-96.0) and negative predictive value was 88.9% (95% CI 83.93-92.46) (Table 2).
Sensitivity according to the two-time intervals from onset of symptoms was 60.0% (95%CI 26.2-87.8) and 73.2% (95%CI 57.1-85.8) at 7-13 and ≥14 days, respectively (Table 3). Specificity at 7-13 and ≥14 days of the onset of symptoms was 97.9% (95%CI 88.7-99.9) and 96.1% (95%CI 89.0-99.2) respectively (Table 3).
The analysis to assess the test accuracy between infants (<2 years old) and older children (≥2 years old) did not show significant differences. LFI test overall sensitivity and specificity for those <2 years old (n=36, PCR+=10) were 70% (95%CI 34.8-93.3) and 100% (95%CI 86.8-100.0) respectively, while for children ≥2 years old (n=139, PCR+= 41) sensitivity was 70.7% (95%CI 54.5-83.9) and specificity 95.9% (95%CI 89.9-98.9).
Discussion
To the authors’ knowledge, this is the first study testing LFI accuracy in the pediatric population in a prospective study during the COVID-19 pandemic. The specificity found was over 96% regardless of days after the onset of symptoms. However, the sensitivity of LFI was low (73.2%), even lower during the second week after disease onset. This finding is in contrast with the commercial test description and illustrates the importance of clinical studies to validate any test in a specific population. Accuracy did not differ between infants and older children.
The authors found a lower sensitivity than in most other reports, but it is important to point out that the performance of these tests may change according to different manufacturers. Previous studies in adults have shown sensitivity numbers around 80%-90%.20,21,22,23 Further, most of these studies evaluated patients with more severe clinical courses and the possible lower levels of antibodies in milder disease may also account for such differences. The exact immunologic specificities between SARS-CoV-2 infected children and adults are still unclear and under investigation. Possible explanations may include a lower expression of angiotensin-converting enzyme 2 receptors, immature humoral and cellular immune responses, and different innate and adaptive responses in children, when compared to adults.24
The rapid point-of-care diagnosis was identified as a research priority by WHO R&D Blue Print expert group.27 Mean days of seroconversion for adult COVID-19 confirmed cases is around 9-12 days post onset.27,28 Some studies describe almost 100% of seroconversion in adult patients within 3 weeks of symptoms onset.29 Although there are no previous reports of the accuracy of serologic LFI in children and adolescents, studies about serology in this age group may be a proxy for the understanding of both tests. Serological diagnosis is especially important for patients who seek health care late for a molecular test.
The results of the present study suggest that a positive result may confirm the diagnosis, but a negative result is not sufficient to rule out the disease in children. In day-to-day practice, the seek for care usually occurs during the first week after the onset of signs and symptoms, when the sensitivity of the serological test is extremely low. Consequently, the LFI has no role in this scenario. If used in epidemiological surveys, the limitations in sensitivity must be considered for the interpretation of such data. It may be useful to evaluate clusters of cases, since a COVID-19 early diagnosis may allow confirmation of subsequent cases by epidemiological criteria. Furthermore, as serological tests are useful as diagnostic criteria for MIS-C, LFI may be helpful in the differential diagnosis of this syndrome at ERs.30 Despite these limitations, LFI testing is simple to perform and has a low cost, with potential use especially in low-resource or remote settings.
This study has some limitations. Firstly, 179 patients did not return for LFI testing. Pediatric patients with COVID-19 usually have a good prognosis and caregivers probably had the concern to return to a hospital setting for further testing when the clinical evolution was mild. Patients who did not return were younger, but the test performance was not different between age groups. A higher proportion of caucasian children did not undergo LFI testing, however, the authors believe that such differences would not change our overall results. Still, individuals who did not return could have a milder clinical course. Although controversial, mild disease has been related to lower levels of antibodies.31 Therefore, due to this subject loss in the study, the sensitivity found could be overestimated. Also, RT-PCR was used as the reference diagnostic method even with limitations such as the timing for collection and a sensitivity around 71%,32 which might have led to an underestimation in specificity and overestimation in sensitivity observed for the LFI test. Another point to address, the RT-PCR sample collection was performed 8 days or later after the onset of symptoms in 28 children, which could result in more false negative RT-PCR results. This might lead to underestimation of the specificity and overestimation of the LFI sensitivity. Finally, in the present study’s sample, COVID-19 prevalence was around 30%. Interpretation of these results must be careful in different epidemiological scenarios, as accuracy (especially PPV and NPV) may change according to prevalence.
Conclusion
Despite the high specificity (>96%) of LFI in a pediatric population, the sensitivity was lower than reports among adults. Moreover, the use of LFI should not be recommended during the first two weeks after the onset of symptoms. Further studies are necessary to assess the long-term performance of this test in children, as it is not known how long antibodies remain positive after an acute infection.
-
FundingThis work was supported by the Ministry of Health of Brazil, through the Institutional Development Program of the Brazilian National Health System (PROADI-SUS) in collaboration with Hospital Moinhos de Vento. The Ministry of Health of Brazil donated the rapid LFI tests used in the study, but had no role in study design; in the collection, analysis and interpretation of data; and in the decision to submit the article for publication.
-
Supplementary materialsSupplementary material associated with this article can be found in the online version at https://doi.org/10.1016/j.jped.2021.04.010.
Acknowledgments
The authors thank the Scientific Committee of the Research Support Nucleus (NAP) of Moinhos de Vento Hospital for technical-scientific consultancies. The authors thank Matias Segelis Vieira, statistician who supported the data analysis, the field workers, laboratory team, and the staff from Hospital Moinhos de Vento, and from Hospital Restinga e Extremo Sul: Aline Andrea da Cunha, Sidiclei Machado Carvalho, Fabio Jose Rockenbach, Kelly Viegas Antunes, Marcelo da Silva Ferreira, Rafael Garcia Trindade, Thayna Silva Lino, Tiago da Silva Silvano, Adriana da Silva Silveira, Alceu Kuckoski, Ana Paula da Silva Lopes, Andreia Escobar da Costa, Clarice Cardoso Machado, Erica Vieira da Silva, Evelin Inacia da Silva, Luciana Rodrigues Ribeiro, Marcely Mayr da Costa, Morgana Thais Carollo Fernandes, Rafael da Silva Cassafuz.
References
- 1. Pollard CA, Morran MP, Nestor-Kalinoski AL. The COVID-19 pandemic: a global health crisis. Physiol Genomics. 2020;52:549–57.
- 2. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020;395:470–3.
- 3. Li R, Pei S, Chen B, Song Y, Zhang T, Yang W, et al. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2). Science. 2020;368:489–93.
- 4. Chu DK, Akl EA, Duda S, Solo K, Yaacoub S, Schünemann HJ, et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. Lancet. 2020;395:1973–87.
- 5. Nicola M, Alsafi Z, Sohrabi C, Kerwan A, Al-Jabir A, Iosifidis C, et al. The socio-economic implications of the coronavirus pandemic (COVID-19): A review. Int J Surg Lond Engl. 2020;78:185–93.
- 6. Pulia MS, O’Brien TP, Hou PC, Schuman A, Sambursky R. Multitiered screening and diagnosis strategy for COVID-19: a model for sustainable testing capacity in response to pandemic. Ann Med. 2020;52:207–14.
- 7. Honein MA, Christie A, Rose DA, Brooks JT, Meaney-Delman D, Cohn A, et al. Summary of guidance for public health strategies to address high levels of community transmission of SARS-CoV-2 and related deaths, December 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1860–7.
- 8. Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L-P, Johnston JC, et al. Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis. BMJ. 2020;370: m25169.
- 9. Mustafa Hellou M, Gorska A, Mazzaferri F, Cremonini E, Gentilotti E, De Nardo P, et al. Nucleic acid amplification tests on respiratory samples for the diagnosis of coronavirus infections: a systematic review and meta-analysis. Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis. 2021;27:341–51.
- 10. Bastos GA, Azambuja AZ de, Polanczyk CA, Graf DD, Zorzo IW, Maccari JG, et al. Clinical characteristics and predictors of mechanical ventilation in patients with COVID-19 hospitalized in Southern Brazil. Rev Bras Ter Intensiva. 2020;32:487–92.
- 11. Hashim MJ, Alsuwaidi AR, Khan G. Population Risk Factors for COVID-19 Mortality in 93 Countries. J Epidemiol Glob Health. 2020;10:204–8.
- 12. Williams N, Radia T, Harman K, Agrawal P, Cook J, Gupta A. COVID-19 Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in children and adolescents: a systematic review of critically unwell children and the association with underlying comorbidities. Eur J Pediatr. 2020; 180:689–97.
- 13. Abrams JY, Godfred-Cato SE, Oster ME, Chow EJ, Koumans EH, Bryant B, et al. Multisystem inflammatory syndrome in children associated with severe acute respiratory syndrome Coronavirus 2: a systematic review. J Pediatr. 2020;226:45–54.e1.
- 14. Feldstein LR, Rose EB, Horwitz SM, Collins JP, Newhams MM, Son MB, et al. Multisystem inflammatory syndrome in U.S. Children and adolescents. N Engl J Med. 2020;383:334–46.
- 15. Ahmed M, Advani S, Moreira A, Zoretic S, Martinez J, Chorath K, et al. Multisystem inflammatory syndrome in children: A systematic review. EClinicalMedicine. 2020;26:100527.
- 16. Heald-Sargent T, Muller WJ, Zheng X, Rippe J, Patel AB, Kociolek LK. Age-related differences in nasopharyngeal severe acute respiratory syndrome coronavirus 2 (sars-cov-2) levels in patients with mild to moderate coronavirus disease 2019 (COVID-19). JAMA Pediatr. 2020;174:902-3.
-
17. Hurst JH, Heston SM, Chambers HN, Cunningham HM, Price MJ, Suarez L, et al. SARS-CoV-2 Infections among children in the biospecimens from respiratory virus-exposed kids (BRAVE Kids) study. medRxiv [Internet]; 2020 Sep 1, [cited 2020 Dec 28]; Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7480040/
» https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7480040/ - 18. Yonker LM, Neilan AM, Bartsch Y, Patel AB, Regan J, Arya P, et al. Pediatric severe acute respiratory syndrome Coronavirus 2 (SARS-CoV-2): clinical presentation, infectivity, and immune responses. J Pediatr. 2020;227:45–52.e5.
- 19. Euser S, Aronson S, Manders I, Lelyveld S van, Herpers B, Sinnige J, et al. SARS-CoV-2 viral load distribution reveals that viral loads increase with age: a retrospective cross-sectional cohort study. medRxiv.; 2021 Jan 22, 2021.01.15.21249691.
- 20. Kohmer N, Westhaus S, Rühl C, Ciesek S, Rabenau HF. Clinical performance of different SARS-CoV-2 IgG antibody tests. J Med Virol. 2020;92:2243–7.
- 21. Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev. 2020;6:CD013652.
- 22. Maine GN, Lao KM, Krishnan SM, Afolayan-Oloye O, Fatemi S, Kumar S, et al. Longitudinal characterization of the IgM and IgG humoral response in symptomatic COVID-19 patients using the Abbott Architect. J Clin Virol. 2020;133:104663.
- 23. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol. 2020;38:1174–83.
- 24. Gupta S, Malhotra N, Gupta N, Agrawal S, Ish P. The curious case of coronavirus disease 2019 (COVID-19) in children. J Pediatr. 2020;222:258–9.
- 25. Mehta P, McAuley DF, Brown M, Sanchez E, Tattersall RS, Manson JJ. COVID-19: consider cytokine storm syndromes and immunosuppression. Lancet. 2020;395:1033–4. 28.
- 26. Yonker LM, Shen K, Kinane TB. Lessons unfolding from pediatric cases of COVID-19 disease caused by SARS-CoV-2 infection. Pediatr Pulmonol. 2020;55:1085–6.
- 27. Peeling RW, Wedderburn CJ, Garcia PJ, Boeras D, Fongwen N, Nkengasong J, et al. Serology testing in the COVID-19 pandemic response. Lancet Infect Dis. 2020;20:e245–9.
- 28. Lou B, Li T-D, Zheng S-F, Su Y-Y, Li Z-Y, Liu W, et al. Serology characteristics of SARS-CoV-2 infection after exposure and post-symptom onset. Eur Respir J. 2020;56:2000763.
- 29. Long Q-X, Liu B-Z, Deng H-J, Wu G-C, Deng K, Chen Y-K, et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med. 2020;26:845-8.
-
30. CDC. Multisystem Inflammatory Syndrome in Children (MIS-C) Associated with Coronavirus Disease 2019 (COVID-19). [cited 2021 Apr 16]. Available from: https://emergency.cdc.gov/han/ 2020/han00432.asp
» https://emergency.cdc.gov/han/ 2020/han00432.asp - 31. Post N, Eddy D, Huntley C, van Schalkwyk MC, Shrotri M, Leeman D, et al. Antibody response to SARS-CoV-2 infection in humans: A systematic review. PloS One. 2020;15:e0244126.
- 32. Long DR, Gombar S, Hogan CA, Greninger AL, O’Reilly-Shah V, Bryson-Cahn C, et al. Occurrence and timing of subsequent severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction positivity among initially negative patients. Clin Infect Dis. 2021;72:323-6.
- 33. Woloshin S, Patel N, Kesselheim AS. False negative tests for SARS-CoV-2 Infection – challenges and implications. N Engl J Med. 2020;383:e38.
- 34. de Abreu MC, C Choquet, Petit H, Bouzid D, Damond F, Marot S, et al. SARS-CoV-2 IGM and IGG rapid serologic test for the diagnosis of COVID-19 in the emergency department. J Infect. 2020;81:816–46.
Publication Dates
-
Publication in this collection
20 Apr 2022 -
Date of issue
Mar-Apr 2022
History
-
Received
05 Feb 2021 -
Accepted
27 Apr 2021 -
Published
03 June 2021