Evaluation of a new brand of immunochromatographic test for visceral leishmaniasis in Brazil made available from 2018

ABSTRACT

Immunochromatographic tests based on the recombinant antigen K39 represent a major advance in diagnosing visceral leishmaniasis (VL) in recent years. Some performance variations are expected and have occurred in the use of several commercial rapid tests, especially in different geographical settings. This is the first evaluation in the Americas of the test recently provided by the public health system in Brazil for the diagnostic of VL, the OnSite™ Leishmania IgG/IgM Combo. In this first clinical test evaluation, 113 VL-positive patient samples and 73 negative controls were tested and a sensitivity of 91.2% and specificity of 94.5% were observed. These results indicate the need for further analysis and comparisons with the performance of other available commercial tests in order to define the impact of this new test on the quality of VL diagnosis in Brazil.

KEYWORDS:
Visceral leishmaniasis; Diagnosis; Rapid test; Antigen rK39

INTRODUCTION

Rapid tests using the recombinant K39 antigen (rK39) represent a breakthrough in visceral leishmaniasis (VL) diagnosis in recent decades. This antigen is a 39-amino acid repeat encoded by a kinesin-related gene of Leishmania chagasi, donovani-specific complex¹. It has been used for detection of antibodies with high sensitivity and specificity, through different platforms. The Kalazar Detect™ (InBios International, USA) was the first rapid test provided by the Brazilian public health system in 2009. After its acquisition, the Kalazar Detect™'s performance was assessed, showing a sensitivity of 88.1% and specificity of 90.6%². In 2015, the Kalazar Detect™ test was replaced by the IT LEISH^® (BIO-RAD Laboratories Inc., France), a test previously validated in Brazil using capillary blood samples and presenting 93% sensitivity and 97% specificity³. In 2016, a study demonstrated that replacing the diagnostic test was economically correct because the IT LEISH^® was a more cost-effective diagnostic strategy than the Kalazar Detect™ in Brazil⁴.

In 2017, another new rapid test became available in Brazil, the OnSite™ Leishmania IgG/IgM Combo test (CTK Biotech, USA). Although the new test has been standardized for use in serum and capillary blood samples, no data are available on its performance for diagnosing VL by L. infantum. Thus, despite being conducted after the purchase and distribution of the test by the public health system, this study aimed to evaluate the performance of the OnSite™ Leishmania IgG/IgM Combo test, for diagnosing VL in Brazil.

MATERIAL AND METHODS

The serum samples used in this study were anonymized and are part of a historical collection of the Instituto Rene Rachou, a reference laboratory for leishmaniasis in Brazil, and their use was approved by the local Research Ethics Committee (Protocol N° 44549915200005091).

Samples of stored sera from 186 patients were tested. The age of patients ranged from 1 month to 70 years and 62.3% of them are men (114/186). None of the patients had a diagnosis of HIV infection. Of these, 113 samples were from patients classified as VL cases (positive samples), whose diagnosis were confirmed by parasitological methods (bone marrow smear or culture). The remaining 73 samples were from patients presenting characteristic signs and symptoms of VL but no Leishmania parasites on parasitological tests and they were diagnosed with other diseases (negative controls). The test was performed according to the manufacturer's instructions and interpreted by three independent observers. Each observer was blinded with respect to the other's readings and to the patient's disease status (positive or negative). The OnSite™ Leishmania IgG/IgM Combo test differs from the other rapid tests because it detects both IgG and IgM antibodies. According to the manufacturer, the test is considered negative when only one red line appears (control line) and positive when the “M line” (indicating IgM antibody presence) or the “G line” (indicating IgG antibody presence) or both lines appear in addition to the control line. To account for all band reactivity possibilities, the analysis was performed considering two scenarios:

Scenario 1: the presence of at least one of the two reactive bands (IgG or IgM) define the test as positive, as recommended by the manufacturer;

Scenario 2: the presence of a reactive IgG band defines the test as positive, independently of the IgM band result.

The test was evaluated using samples that were previously used to evaluate the IT LEISH^® test a few years ago³; however, some patient's samples were no longer available. Thus, two strategies were used to analyze the OnSite™ Leishmania IgG/IgM Combo's performance: i) its performance was compared with that of IT LEISH^®, considering only the samples tested for both tests; and ii) its performance was compared with that of IT LEISH^® as reported for a larger sample by Machado de Assis et al.³.

In this analysis, the performance measures of interest were sensitivity, specificity and accuracy. Analyses were performed in the Open Source Epidemiologic Statistics for Public Health (OpenEpi) version 2.3⁵. Exact binomial 95% confidence limits (95% CI) were calculated for each performance measure. The comparison of OnSite™ Leishmania IgG/IgM Combo performance with the previously described performance of IT LEISH^® was carried out by evaluating the confidence interval of the difference between the tests (using the chi-square test). The Kappa index was interpreted following criteria of Landis and Koch⁶ to assess concordance among the observers and between the OnSite™ Leishmania IgG/IgM Combo and the IT LEISH^® tests. A P value < 0.05 was considered statistically significant.

RESULTS

The OnSite™ Leishmania IgG/IgM Combo rapid test's performance for the two proposed scenarios is presented in Table 1. Using the positivity criteria recommended by the manufacturer, the estimated sensitivity of the OnSite™ Leishmania IgG/IgM Combo was 91.2% (95% CI: 84.5-95.1%), which was the same sensitivity observed in Scenario 2 since no patient in the VL group presented isolated reactivity regarding the IgM band. The presence of the reactive “M line” was observed in only one sample in which no reactive “G line” was present (case details: a 28-year-old patient, negative by parasitological tests, was IT LEISH^® negative and diagnosed with mycobacteriosis). Seven samples that tested positive by the OnSite™ Leishmania IgG/IgM Combo showed a low intensity band that required an external light source (flashlight). These samples were considered positive following the manufacturer's instructions. Of these seven results, six were true-positives (samples from VL patients). Conversely, ten samples with a positive parasitological test were negative on the OnSite™ Leishmania IgG/IgM Combo, and of these, eight tested positive by IT LEISH^®. These samples were rescreened with IT LEISH^®, and the previous positive results were confirmed.

Table 1
Performance of the OnSite™ Leishmania IgG/IgM Combo test

Studygroup	Scenario 1*			Scenario 2**
	Positive result	Negative result	Total	Positive result	Negative result	Total
Case	103	10	113	103	10	113
Non-case	4	69	73	3	70	73
Total	107	79	186	106	80	186
Sensitivity (%) (95%CI)		91.2 (84.5 – 95.1)			91.2 (84.5 – 95.1)
Specificity (%) (95%CI)		94.5 (86.7 – 97.9)			95.9 (88.6 – 98.6)
Accuracy (%) (95%CI)		92.5 (87.8 – 95.5)			93.0 (88.4 – 95.9)

• ^*
  Scenario 1: The test was considered positive when there is at least one of the reactive bands (IgG or IgM), as recommended by the manufacturer;
• ^**
  Scenario 2: The test was considered positive when the reactive IgG band is present, independently of the IgM band result.
• 95%CI: confidence interval of 95%.

Considering the positivity based on the presence of any reactive band, of the 73 negative samples, the OnSite™ Leishmania IgG/IgM Combo exhibited four false-positive results and a specificity of 94.5% (95% CI: 86.7-97.9%). These four samples (apparently false-positives on the OnSite™ test) were diagnosed as follows: bacterial sepsis, hepatic insufficiency, lymphoma and mycobacteriosis. Of these, two samples screened positive on the IT LEISH^®, and one presented only the reactive “M line” on the OnSite™ Leishmania IgG/IgM Combo. Considering the test positivity as the IgG band presence, three false-positive results on the OnSite™ Leishmania IgG/IgM Combo test was observed, a specificity of 95.9% (95% CI: 88.6-98.6%).

Comparing the IT LEISH^® and the OnSite™ Leishmania IgG/IgM Combo tests on the same tested samples, no significant differences were observed in sensitivity or specificity. The concordance rate between the OnSite™ Leishmania IgG/IgM Combo and the IT LEISH^® by the Kappa index was 0.88 (95% CI: 0.81-0.95) for Scenario 1 and 0.89 (95% CI: 0.82-0.96) for Scenario 2. In both scenarios, the observers were in 100% agreement on the test interpretation (K = 1.0). Compared with the previously published IT LEISH^® test performance³, the specificity of the OnSite™ Leishmania IgG/IgM Combo was lower (94.5%; 95% CI: 86.7-97.9% versus 97%; 95% CI: 91.6-99%, p=0.04), with no difference in sensitivity between the tests.

DISCUSSION

No consensus exists on the minimum performance parameters required for VL diagnostic tests. In addition to the severity of each disease, which determines the acceptable tolerance for misdiagnosis, this definition depends mainly on the level of technological development previously achieved for a given methodology, as reflected in the performance of similar products that are currently available. In VL cases, an ideal test meets the highest performance compared with others, along with affordable cost and operability, when it is feasible in places with low infrastructure in underdeveloped countries where the disease predominates. According to Boelaert et al.⁷, the minimum sensitivity and specificity rates required for a VL rapid diagnostic test would be 95% and 98%, respectively. However, there is no consensus on the minimum accepted performance for a VL screening test. In Brazil, for example, in the public call for the purchase of the new rapid test for VL, the minimum performance required for both sensitivity and specificity was 90%⁸. Considering 95% for both sensitivity and specificity and the manufacturer's positivity criteria for the OnSite™ Leishmania IgG/IgM Combo, the test performance would be considered unsatisfactory, while accepting 90% for the minimum required parameters, the same performance would be considered within the desired limits.

The manufacturer of OnSite™ Leishmania IgG/IgM Combo defines the test positivity based on either of the two bands (IgM or IgG), but this instruction could pose a problem. In the present study, IgM band reactivity was observed in only one case in which the reactive M line was a false-positive reaction in the serum sample from a patient clinically diagnosed with mycobacteriosis. The potential consequence of this observation, and the lower specificity observed for the OnSite™ Leishmania IgG/IgM Combo test compared with the VL rapid test previously available in Brazil (IT LEISH^®), would be the increment of false VL diagnoses, that could lead to unnecessarily patients exposition to toxic leishmaniasis treatment options. For an indirect comparative analysis, it is important to consider data from some studies that evaluated the performance of rapid tests for VL in Brazil, published in recent years. The sensitivity and specificity reported for these tests were respectively: Kalazar Detect™: 88.1-95.5% and 90.6-100%^2,9,10; IT LEISH^®:93-100% and 96.5-100%^2,3,11 and OrangeLife^® (OrangeLife, Brazil): 88.9 and 94.7%¹⁰.

Globally, results of the World Health Organization's initiative addressing the performance of the available commercial rapid VL tests containing bound rK39 or rKE16 antigen in endemic regions should be considered¹². On the Indian subcontinent, all tests performed well, with high sensitivity, ranging from 92.8% (95% CI: 88.9-95.4%; Crystal^® KA) to 100% (95% CI: 97.9-100%; Signal^® KA) and high specificity, ranging from 96.0% (92.8%–97.8%; Kalazar Detect™) to 100% (95% CI: 97.8-100%; Signal^® KA)¹². However, in East Africa and Brazil, lower sensitivity was observed, ranging from 36.8% (95% CI: 31.1-42.9%; Crystal^® KA) to 92% (95% CI: 87.8-94.8%; IT LEISH^®)¹². IT LEISH^®'s sensitivity was significantly better than any other products evaluated in East Africa (p< 0.0001) and Brazil (p= 0.013). The OnSite™ Leishmania IgG/IgM Combo test was not included in this global assessment while another test produced by the same manufacturer, the OnSite™ Leishmania Ab Rapid test, was evaluated only in India using samples from 250 VL-patients and 249 non-VL, with 99.6% sensitivity and 96.8% specificity¹². The sensitivity and specificity reported here with the OnSite™ Leishmania IgG/IgM Combo test were lower than those previously reported in India. The differences may be due to antigen preparations, parasite genetic profiles prevalent in both regions and the population's genetic diversity. The “OnSite™ Leishmania Ab Rapid” test as well as “Onsite™ Leishmania IgG/IgM Combo” test detects IgG and IgM antibodies. The difference is that the test evaluated here detects IgM and IgG in separate lines, while the other test detects both antibodies in the same line without distinction of these two subclasses of antibodies.

Although, both OnSite™ Leishmania IgG/IgM Combo and IT LEISH^® are lateral flow immunochromatographic rapid tests based on the rK39, there are some structural differences in the cassettes. In the Onsite™ Leishmania IgG/IgM Combo, colloidal gold nanoparticles are functionalized with the rK39 while anti-human IgG and anti-human IgM antibodies constitute test lines on nitrocellulose membrane. On the other hand, in the IT LEISH^® test, the rK39 antigen is placed on the test line while colloidal gold nanoparticles are functionalized with anti-human IgG antibodies. These differences could explain the variation observed between these two tests.

Finally, rapid diagnostic tests may improve early VL detection, but their real-world performance varies, requiring local validation. In this first evaluation of the OnSite™ Leishmania IgG/IgM Combo in Brazil, the test performed satisfactorily, if minimum parameters are accepted. The presence of the IgM band, an innovation among the rapid tests for VL, at least in this study, was not associated with increased sensitivity, but with a false-positive reaction, thus requiring caution in its interpretation.

• FINANCIAL SUPPORT
  Conselho Nacional de Desenvolvimento Científico e Tecnológico – CNPq (Grant N° 310938/2017-1 to Ana Rabello).

ACKNOWLEDGMENT

We thank the Ministry of Health of Brazil, which acquired the tests that were assigned by Instituto Octavio Magalhães, Fundação Ezequiel Dias (FUNED) for this study.

REFERENCES

• ¹
  Burns JM Jr, Schreffler WG, Benson DR, Ghalib HW, Badaro R, Reed SG. Molecular characterization of a Kinesis-related antigen of Leishmania chagasi that detects specific antibody in African and American visceral leishmaniasis. Proc Natl Acad Sci U S A. 1993;90:775-9.
• ²
  Peruhype-Magalhães V, Machado de Assis TS, Rabello A. Use of the Kala-Azar Detect^® and IT-LEISH^® rapid tests for the diagnosis of visceral leishmaniasis in Brazil. Mem Inst Oswaldo Cruz. 2012;107:951-2
• ³
  Machado de Assis TS, Braga AS, Pedras MJ, Barral AM, Siqueira IC, Costa CH, et al. Validação do teste imunocromatográfico rápido IT-LEISH^® para o diagnóstico da leishmaniose visceral humana. Epidemiol Serv Saude. 2008;17:107-16.
• ⁴
  Machado de Assis TS, Azeredo-da-Silva AL, Werneck GL, Rabello A. Cost-effectiveness analysis of diagnostic tests for human visceral leishmaniasis in Brazil. Trans R Soc Trop Med Hyg. 2016;110:464-71.
• ⁵
  Dean AG, Sullivan KM, Soe MM. OpenEpi: Open source epidemiologic statistics for public health. [cited 2018 May 15]. Available from: http://www.openepi.com
  » http://www.openepi.com
• ⁶
  Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33:363-74.
• ⁷
  Boelaert M, Bhattacharya S, Chappuis F, El Safi SH, Hailu A, Mondal D, et al. Evaluation of rapid diagnostic tests: visceral leishmaniasis. Nat Rev Microbiol. 2007;5:S30-9.
• ⁸
  Brasil. Ministério da Saúde. Nota informativa N° 3/2018-CGLAB/DEVIT/SVS/ MS. [cited 2018 Jun 14]. Available from: http://portalarquivos2.saude.gov.br/images/pdf/2018/fevereiro/26/NOTA-INFORMATIVA-n3.pdf
  » http://portalarquivos2.saude.gov.br/images/pdf/2018/fevereiro/26/NOTA-INFORMATIVA-n3.pdf
• ⁹
  Carvalho SF, Lemos EM, Corey R, Dietze R. Performance of recombinant K39 antigen in the diagnosis of Brazilian visceral leishmaniasis. Am J Trop Med Hyg. 2003;68:321-4.
• ¹⁰
  da Silva MR, Brandão NA, Colovati M, de Sousa MM, de Lima LC, Dorta ML, et al. Performance of two immunochromatographic tests for diagnosis of visceral leishmaniasis in patients coinfected with HIV. Parasitol Res. 2018;117:419-27.
• ¹¹
  Amato Neto V, Amato VS, Tuon FF, Gakiya E, de Marchi CR, de Souza RM, et al. False-positive results of a rapid K39-based strip test and Chagas disease. Int J Infect Dis. 2009;13:182-5.
• ¹²
  Cunningham J, Hasker E, Das P, El Safi S, Goto H, Mondal D, et al. A global comparative evaluation of commercial immunochromatographic rapid diagnostic tests for visceral leishmaniasis. Clin Infect Dis. 2012;55:1312-9

Publication Dates

• Publication in this collection
  13 Sept 2018
• Date of issue
  2018

History

• Received
  17 May 2018
• Accepted
  04 July 2018