Analytical evaluation of thirty-two severe acute respiratory syndrome 2 lateral flow antigen tests demonstrates sensitivity remains with the SARS-CoV-2 Gamma lineageABSTRACT
Background: The emergence of variants of concern (VOCs) requires an ongoing assessment of the performance of antigen lateral flow tests (Ag-RDTs). The limit of detection (LOD) of 32 Ag-RDTs was evaluated using the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Gamma variant.
Methods: Ag-RDTs were performed according to the manufacturer’s instructions with a clinical isolate of the Gamma variant.
Results: Twenty-eight of the 32 Ag-RDTs exceeded the World Health Organization criteria.
Conclusions: This comprehensive analytical evaluation of Ag-RDTs demonstrated that the test performance was maintained with Gamma VOC.
Keywords: SARS-CoV-2; SARS-CoV-2 Gamma variant; Variant of concern; Diagnostic tests
The emergence of the variant of concern (VOC) Gamma (Pango P.1) began in November 2020 in the state of Amazonas, Brazil. The Gamma variant was estimated to be 1.7 to 2.4 times more transmissible than other local strains in Brazil1 and quickly started to be detected at increasing rates from January 2021 onwards throughout the country. As a result, this variant has become the predominant lineage associated with the second wave of infections, with over 13 million confirmed cases and 350,000 deaths2.
As of January 10, 2022, the Gamma strain had spread to 93 countries and remained one of the most prevalent variants circulating in the Americas, with the highest frequency in South America. In South America, the proportion of Gamma-associated cases ranges from 5% to 100%, depending on the country. Countries with higher levels of circulation of Gamma variants include Saint Vincent and Grenadines (100%), Haiti (100%), Trinidad and Tobago (50%), Argentina (30%), and Venezuela (30%), followed by Chile, Brazil, Peru, Ecuador, and Suriname, with 10%-20% of the circulating severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants being Gamma3.
The use of antigen SARS-CoV-2 lateral flow diagnostic tests (Ag-RDTs) has become one of the first lines of defense against coronavirus disease 2019 (COVID-19)4. Ag-RDTs have been shown to be accurate in detecting the vast majority of individuals with a high viral load. In addition, they can determine the presence of SARS-CoV-2 antigens in clinical samples in 10-30 min and have facilitated the early identification and isolation of cases5,6. In turn, this has slowed transmission, enabled the provision of targeted care, and helped protect health systems7.
Gamma has 21 mutations, including 10 in spike (S) and 3 in nucleocapsid (N) proteins. As SARS-CoV-2 Ag-RDTs target S or N proteins, there is a concern that these mutations could affect Ag-RDT performance. Thus, we evaluated the limit of detection (LOD) of 32 commercially available Ag-RDTs using Gamma VOC and compared the results with LODs previously determined with the Alpha (B.1.1.7) and ancestral (B.1) lineages7,8.
A clinical isolate of the Gamma lineage of SARS-CoV-2 (hCoV-19/Japan/TY7-503/2021) was used for this evaluation. This clinical isolate was obtained from the BEI resources after isolation from a SARS-CoV-2 positive passenger from Brazil in an airport quarantine in Japan in January 2021. The virus stock was propagated into Vero E6 cells (C1008; African green monkey kidney cells), which were maintained at 37 °C with 5% CO2 in Dulbecco’s modified Eagle’s medium (DMEM) supplemented with 4.5 g/L glucose and L-glutamine (Lonza, US), 10% fetal bovine serum (Sigma, US), and 50 units/mL of penicillin/streptomycin (Gibco, US).
To determine the LOD of the 32 Ag-RDTs, a fresh aliquot of the third passage of the virus was serially diluted in DMEM from 1.0x105 to 1.0x102 plaque-forming units (pfu)/mL and tested as a direct culture. The viral dilutions were added directly at a 1:10 ratio to the respective Ag-RDT extraction buffers. Ag-RDTs were performed in accordance with the manufacturer’s instructions. Each dilution was tested in triplicate, with Ag-RDT extraction buffer spiked in a 1:10 ratio, with DMEM acting as a negative control. When a 10-fold LOD was found, two-fold dilutions were made and tested to confirm the lowest LOD (LLOD). The LOD was defined as the last dilution of a valid test in which all three replicates were positive. The presence of a control line determined the validity, and only valid tests were included in the analysis. A positive result was interpreted visually by two operators based on the presence of a test line of any intensity. In the event of a discordant result, a third operator read the test and acted as a tiebreaker. All experimental procedures were performed in a containment level 3 laboratory under biosafety level 3 conditions.
Frozen aliquots of the third passage of the virus were quantified by plaque assay as previously described9 to determine pfu/mL (gcn/mL). To estimate the genome copy number of each serial dilution, ribonucleic acid (RNA) was extracted using a QIAMP Viral RNA mini kit (Qiagen, Germany), and each viral dilution was tested in triplicate using the COVID-19 Genesig RT-qPCR kit (PrimerDesign, UK). The gcn/mL was calculated from the mean cycle threshold (Ct) values of these replicates. An RNA standard curve was generated by testing five replicates of each 10-fold serial dilution of SARS-CoV-2 synthetic RNA (PrimerDesign, UK). The positive control contained a standard number of copies of the SARS-CoV-2 RNA sequence. Quantitative reverse transcription-polymerase chain reaction (RT-qPCR) was performed using Rotor-Gene Q (Qiagen, Germany).
Once the LOD was determined for each of the 32 Ag-RDTs, the LOD of each Ag-RDT with the Gamma lineage was compared with the ancestral and Alpha SARS-CoV-2 lineages using data from a previous work5. Finally, LODs were compared using the Kruskal-Wallis test (IBM SPSS Statistics v28.0). The significance level was set at P < 0.05.
We found that 21/32 Ag-RDTs had an analytical LOD ≤ 5.0x102 pfu/mL (ActiveXpress, Bioperfectus, Core Test, Espline, Genedia, Fortress, iChroma, InTec, Joysbio, LumiraDx, Nadal, NowCheck, Panbio, PerkinElmer, RightSign, Roche, Standard F, Standard Q, Strong Step, Sure-Status, and Wantai), fulfilling the acceptable criteria of the British Department of Health and Social Care (DHSC). Additionally, 28/32 (including Biocredit, Covid-go, Excalibur, Mologic, Tigsun, and Wondfo) had a LOD ≤ 1.0x106 gcn/mL, fulfilling the recommendations of the WHO Target Product Profile for SARS-CoV-2 Ag-RDT (Table 1). The most sensitive tests with the Gamma variant were the Core Test and InTec, both of which had analytical LOD of 1.0x101 pfu/mL. The least sensitive tests with the Gamma variant were Innova, Flowflex, Hotgen, Innova, Onsite, and RespiStrip, which had a LOD ≥ 2.5x103 pfu/mL.
Description of the Ag-RDTs evaluated in this study and their limit of detection (LOD) using Gamma as variant of concern
Of the Ag-RDTs tested with the ancestral lineage, 18/32 had an analytical LOD ≤ 5.0x102 pfu/mL, and 20/32 had a LOD ≤ 1.0x106 gcn/mL. Compared with the Gamma lineage, 15/32 of the Ag-RDTs tested with the ancestral lineage had a lower analytical LOD, 13/32 had a higher LOD, and 4/32 were equal. Of the Ag-RDTs tested with the Alpha lineage, 25/32 Ag-RDTs had an analytical LOD ≤ 5.0x102 pfu/mL, and 30/32 had a LOD ≤ 1.0x106 gcn/mL. Compared with the Gamma lineage, 16/32 Ag-RDTs tested with the Alpha lineage had a lower analytical LOD, 8/32 had a higher LOD, and 8/32 were equal. Six out of 31 Ag-RDTs had a higher sensitivity for the Gamma lineage than the other two lineages (Biocredit, Core Test, Genedia, InTec, Standard F, and Standard Q). Conversely, 5/32 and 9/32 Ag-RDTs showed a higher sensitivity for Alpha and ancestral lineages, respectively, compared with the other two lineages. In total, 12/32 Ag-RDTs met the acceptance criteria for the DHSC and the recommendations of the WHO Target Product Profile for SARS-CoV-2 Ag-RDTs for all three lineages (ActivXpress, Espline, Fortress, iChroma, Joysbio, Lumira, Nadal, NowCheck, Panbio, Roche, Standard Q, and Sure Status). However, there was no significant difference between the Ag-RDT LOD performance with Gamma compared with either the Alpha (Kruskal-Wallis P = 0.315) or ancestral lineage (Kruskal-Wallis P = 0.378) (Figure 1).
Ag-RDTs have been proven to be essential in our response to the ongoing global SARS-CoV-2 pandemic. Available clinical evaluation data have demonstrated that Ag-RDTs can accurately detect the majority of individuals with high-viral loads7. However, most Ag-RDT validation studies were completed prior to the emergence of variants, and to date, there are limited data on the performance of diagnostics using VOCs10. As the pandemic progresses, VOCs continue to emerge, outcompete earlier lineages and dominate different areas worldwide. It is important to assess the ability of Ag-RDTs to detect newly emerging variants; therefore, infection prevention and control measures can be updated.
Our results continue to build upon previous work7,8,10 and demonstrate that a selection of commercially available Ag-RDTs can detect the Gamma variant, with some presenting an equivalent performance to the ancestral and Alpha lineages. Despite the observed differences in clinical sensitivities, the Ag-RDTs evaluated here were able, on balance, to detect the ancestral, Alpha, and Gamma lineages.
The Gamma variant has three mutations in the N protein, which appears to be insufficient to disrupt the highly specific antigen-antibody binding reactions that enable the detection of the SARS-CoV-2 antigen. In addition, the N protein has a relatively low mutation rate7; therefore, more conserved across different variants. Thus, it has been hypothesized that Ag-RDTs targeting the N protein can detect all known variants7. All the Ag-RDTs evaluated in this study targeted the N protein, which contains fewer mutations than the S protein in the Gamma lineage. This may explain the similarities in test performances between the Gamma, Alpha, and ancestral lineages.
Tests targeting the S antigen are expected to have greater difficulty detecting VOCs because of the higher number of mutations in this protein. As Ag-RDTs that target the S protein were not assessed here, we cannot comment on their performance. Further investigations are required to assess the ability of Ag-RDTs to target the S protein to detect VOCs with any sensitivity loss on emerging variants.
Comparing the sensitivities of the Ag-RDTs across different lineages allows continued monitoring of Ag-RDT performance, which could provide early warning signs of a decreased ability to detect VOCs. Twenty-eight out of 32, 20/32, and 30/32 Ag-RDTs met the WHO Target Product Profile for SARS-CoV-2 Ag-RDTs recommendations for the Gamma, Alpha, and ancestral lineages, respectively. This suggests that, despite the comparable performance between different lineages, the differences in LODs have an impact on whether Ag-RDTs satisfy the established criteria. Consequently, some Ag-RDTs may not be suitable for population screening because of their reduced ability to detect certain lineages.
Although the direct application of the cultured virus enabled the evaluation of Ag-RDT analytical sensitivity, it did not replicate the sampling and testing conditions in which the Ag-RDTs were used. Clinical evaluations using patient samples should also be conducted to assess the test performance. Furthermore, the results presented here do not guarantee the ability of the assessed Ag-RDTs to detect other or future VOCs; other strains or variants may contain different N protein mutations that can disrupt detection. As new lineages continue to emerge, there is a need to reassess the ability of Ag-RDTs to diagnose SARS-CoV-2.
One of the limitations of this study is that for practicality, the experiments with the ancestral and Alpha lineages were not performed in parallel to Gamma, and comparative analyses relied on previously published data. Another limitation is that the LOD data were not complemented with accuracy using clinical samples, correlating with Ct values, days of symptom onset, and disease severity, which may interfere with test performance; however, clinical evaluation was outside the scope of this study.
This is the most comprehensive analytical evaluation of COVID-19 Ag-RDTs with Gamma VOC, and we have demonstrated that the test performance is maintained, indicating Ag-RDT compatibility. This evidence supports their continued usage in countries where Gamma strain is circulating. However, clinical diagnostic evaluations in prospective cohorts from these localities are required to provide definitive data on their clinical performance. Ag-RDTs that target S may have different effects. Diagnostic evaluations must continue to monitor the test performance in emerging variants to ensure continued diagnostic performance.
REFERENCES
-
1 Duong D. Alpha, Beta, Delta, Gamma: What’s important to know about SARS-CoV-2 variants of concern? CMAJ. 2021;193(27):E1059-E1060. Available from: https://doi.org/10.1503/cmaj.1095949.
» https://doi.org/10.1503/cmaj.1095949 -
2 Nonaka CKV, Gräf T, Barcia CAL, Costa VF, de Oliveira JL, Passos RH, et al. SARS-CoV-2 variant of concern P.1 (Gamma) infection in young and middle-aged patients admitted to the intensive care units of a single hospital in Salvador, Northeast Brazil, February 2021. Int J Infect Dis. 2021;111:47-54. Available from: https://doi.org/10.1016/j.ijid.2021.08.003.
» https://doi.org/10.1016/j.ijid.2021.08.003 -
3 Global Initiative on Sharing All Influenza Data>, (GISAID). GISAID. https://www.gisaid.org/hcov19-variants/
» https://www.gisaid.org/hcov19-variants/ -
4 World Health Organization (WHO). SARS-CoV-2 antigen-detecting rapid diagnostic tests: an implementation guide: web annex: country readiness checklist for SARS-CoV-2 antigen RDT implementation. World Health Organization. 2020. Available in Available in https://apps.who.int/iris/handle/10665/337937 Accessed in December, 2021.
» https://apps.who.int/iris/handle/10665/337937 -
5 Wagenhäuser I, Knies K, Rauschenberger V, Eisenmann M, McDonogh M, Petri N, et al. Clinical performance evaluation of SARS-CoV-2 rapid antigen testing in point of care usage in comparison to RT-qPCR. EBioMedicine. 2021;69:103455. Available from: https://doi.org/10.1016/j.ebiom.2021.103455.
» https://doi.org/10.1016/j.ebiom.2021.103455 -
6 Cubas-Atienzar AI, Bell F, Byrne RL, Buist K, Clark DJ, Cocozza M, et al. Accuracy of the Mologic COVID-19 rapid antigen test: a prospective multi-centre analytical and clinical evaluation. Wellcome Open Res. 2021;6:132. Available from: https://doi.org/10.12688/wellcomeopenres.16842.1.
» https://doi.org/10.12688/wellcomeopenres.16842.1 -
7 Cubas-Atienzar AI, Kontogianni K, Edwards T, Wooding D, Buist K, Thompson CR, et al. Limit of detection in different matrices of 19 commercially available rapid antigen tests for the detection of SARS-CoV-2. Sci Rep. 2021;11(1):18313. Available from: https://doi.org/10.1038/s41598-021-97489-9.
» https://doi.org/10.1038/s41598-021-97489-9 -
8 Kontogianni K, Cubas-Atienzar AI, Wooding D, Buist K, Thompson CR, Williams CT, et al. Lateral flow antigen tests can sensitively detect live cultured virus of the SARS-CoV-2 B1.1.7 lineage. J Infect. 2021;83(1):e1-e4. Available from: https://doi.org/10.1016/j.jinf.2021.05.033.
» https://doi.org/10.1016/j.jinf.2021.05.033 -
9 Edwards T, Kay GA, Aljayyoussi G, Owen SI, Harland AR, Pierce NS, et al. SARS-CoV-2 Transmission Risk from sports Equipment (STRIKE). medRxiv. 2021. Available from: https://doi.org/10.1101/2021.02.04.21251127.
» https://doi.org/10.1101/2021.02.04.21251127 -
10 Bekliz M, Adea K, Essaidi-Laziosi M, Sacks JA, Escadafal C, Kaiser L, et al. SARS-CoV-2 rapid diagnostic tests for emerging variants. Lancet Microbe. 2021;2(8):e351. Available from: https://doi.org/10.1016/S2666-5247(21)00147-6.
» https://doi.org/10.1016/S2666-5247(21)00147-6
Publication Dates
-
Publication in this collection
06 June 2022 -
Date of issue
2022
History
-
Received
12 Jan 2022 -
Accepted
30 Mar 2022
