INTRODUCTION
The signs and symptoms of visceral leishmaniasis (VL) include fever of more than 2 weeks of duration, anorexia, substantial weight loss, anaemia, enlarged lymph nodes, spleen and liver, as well as pancytopenia (low blood cell count). All of these are unspecific and often confused with symptoms presented by other infectious and non-infectious diseases. Therefore, it is important that clinical examinations are complemented by laboratory tests. Although the detection of the parasite in smears and/or cultures of blood, bone-marrow, lymph nodes or spleen is still considered a reference test, the sensitivity of these techniques varies depending on the biological sample examined. The examination of splenic samples, for example, provides high sensitivity (93–99%) to VL, but the invasiveness and potentially fatal complications associated with splenic aspiration have restricted its indication. On the other hand, the aspiration and examination of bone marrow aspirate, which is recommended due to its safety, presents lower sensitivity (53–86%) (Ministério da Saúde, 2006).
Considering the limitations of the parasitological examinations and given the high levels of circulating antibodies in VL patients, different serological tests have been developed for VL diagnosis, including the immunofluorescent antibody test (IFAT) (Duxbury and Sadun, Reference Duxbury and Sadun1964), the direct agglutination test (DAT) (El Harith et al. Reference El Harith, Kolk and Kager1986; Meredith et al. Reference Meredith, Kroon, Sondorp, Seaman, Goris, Van Ingen, Oosting, Schoone, Terpstra and Oskam1995), and the ELISA (enzyme-linked immunosorbent assay) using crude antigen (Ho et al. Reference Ho, Leeuwenburg, Mbugua, Wamachi and Voller1983), exo-antigens (Martin et al. Reference Martin, Thuita-Harum, Adoyo-Adoyo and Wasunna1998; Rajasekariah et al. Reference Rajasekariah, Ryan, Hillier, Yi, Stiteler, Cui, Smithyman and Martin2001; Ryan et al. Reference Ryan, Smithyman, Rajasekariah, Hochberg, Stiteler and Martin2002) or recombinant K39 (rK39) (Burns et al. Reference Burns, Shreffler, Benson, Ghalibt, Badaro and Reed1993; Badaró et al. Reference Badaró, Benson, Eulálio, Freire, Cunha, Netto, Pedral-Sampaio, Madureira, Burns, Houghton, David and Reed1996). The use of rK39 in the immunochromatographic test represented an improvement in VL diagnosis (Sundar et al. Reference Sundar, Reed, Singh, Kumar and Murray1998). This test, better known as the rK39 rapid test, is easy to use in field conditions being also highly sensitive and specific.
Since 2009, the Brazilian Ministry of Health (MoH) has recommended the use of rK39 rapid test to investigate patients with clinically suspected VL. However, since 2012, the MoH instructs that any case showing undetermined or negative results should be subjected to an IFAT analysis at a Central Public Health Laboratories of the National System of Laboratories (LACENs/SISLAB) or at a reference laboratory (Ministério da Saúde, 2012). The IFAT requires fluorescence microscopes and relatively well-equipped laboratories, making the fulfilment of the MoH's resolution a challenge considering the public health resources. Moreover, IFAT is inadequate for use as a VL confirmatory test since it has a less than ideal sensitivity varying between 86 and 92% and specificity ranging from 81 to 88% (Machado de Assis et al. Reference Machado de Assis, Braga, Pedras, Barral, Siqueira, Costa, Costa, Holanda, Soares, Bia, Caldas, Romero and Rabello2008; Pedras et al. Reference Pedras, Viana, Oliveira and Rabello2008).
The DAT is a speedier and simpler alternative for VL diagnosis than IFAT, as it does not require complex laboratory facilities. Importantly, previous studies revealed that DAT has high sensitivity and specificity for VL diagnosis (Mandal et al. Reference Mandal, Khurana, Dubey, Bhatia, Varma and Malla2008; Cañavate et al. Reference Cañavate, Herrero, Nieto, Cruz, Chicharro, Aparicio, Mulugeta, Argaw, Blackstock, Alvar and Bern2011; Machado de Assis et al. Reference Machado de Assis, Braga, Pedras, Oliveira, Barral, Siqueira, Costa, Costa, Holanda, Soares, Biá, Caldas, Romero and Rabello2011; Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013). The WHO currently recommends that a DAT or a parasitological test should be performed whenever a negative rK39 rapid test result is obtained in a suspected patient from areas where the rK39 rapid test presents sensitivity below than 90% (WHO, 2010). In this context, we developed a freeze-dried DAT antigen produced with Leishmania (L.) infantum promastigotes to compose a prototype kit for VL diagnosis, named DAT-LPC. Different from the DAT antigen produced by two European Institutions (Institute of Tropical Medicine, Antwerp, Belgium and KIT-Biomedical Research, Amsterdam, Holland), the DAT-LPC is produced with L. infantum promastigotes, does not use 2-mercaptoethanol in the samples diluent and the results can be read after 4 h of incubation (in contrast with the 18 h incubation required by these European antigens). In a phase I validation study the DAT-LPC presented sensitivity of 99%, specificity of 98·2% and diagnostic validity of 98·6% importantly when these measurements were compared with those obtained with the DAT-KIT, developed by the Royal Tropical Institute of Amsterdam there was no statistical difference between them (P < 0·05) (Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013). Despite of the high diagnostic performance shown by the DAT-LPC in previous studies (Cota et al. Reference Cota, de Sousa, Freitas Nogueira, Gomes, Oliveira, Assis, Mendonça, Pinto, Saliba and Rabello2013; Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013) a work evaluating its reproducibility was still lacking. Here we assessed the reproducibility and diagnostic accuracy of the DAT-LPC prototype kit in laboratories from four different Brazilian public institutions, using a proficiency panel.
MATERIAL AND METHODS
Participant laboratories
A partnership among four different laboratories in Brazil was established for this study.
Laboratory 1 (reference laboratory)
The Group of Clinical Research and Public Policies in Infectious and Parasitic Diseases at the Centro de Pesquisas René Rachou, Oswaldo Cruz Foundation (CPqRR/FIOCRUZ) was designated as Laboratory 1. Laboratory 1 that provided the proficiency panel is the place where the DAT-LPC was developed.
Laboratory 2
The Parasitic Diseases Service at Ezequiel Dias Foundation is a National Diagnosis Reference for VL in Brazil. The service is responsible for quality control of VL diagnosis performed by the Central Public Health Laboratories of the National System of Laboratories (LACENs/SISLAB), which routinely performs diagnosis of VL using IFAT, rK39 rapid test and polymerase chain reaction (PCR) of samples collected in public laboratories and hospitals.
Laboratory 3
The Laboratory of the Eduardo de Menezes Hospital of the Fundação Hospitalar do Estado de Minas Gerais (FHEMIG) runs tests of inpatients at this hospital, which cares for adult patients with infectious diseases referred by the Brazilian Universal Health System (Sistema Único de Saúde – SUS).
Laboratory 4
The Laboratory of the Health Secretary at Ribeirão das Neves (ML/RN) is a municipal public laboratory that runs examinations of outpatients who are referred from 61 primary health care units, one outpatient clinic for infectious and parasitic diseases, two emergency units and five basic referral units.
Calculating sample size
To determine the sensitivity, specificity and reproducibility with a 95% confidence interval of the DAT-LPC following its use in laboratories from different public health institutions, we first calculated the number of positive and negative serum samples necessary to assess these characteristics using the following equation (Banoo et al. Reference Banoo, Bell, Bossuyt, Herring, Mabey, Poole, Smith, Sriram, Wongsrichanalai, Linke, O`Brien, Perkins, Cunningham, Matsoso, Nathanson, Olliaro, Peeling and Ramsay2010):
$$n{\rm \;\ge \; }z^2 \times p(1 - p)/x^2$$
where n, number of positive or negative samples; z, 0·196; p, sensitivity (or specificity) and x 2, confidence interval (CI). Considering a 95% CI and sensitivity and specificity of 0·99 and 0·98 as previously reported for DAT-LPC (Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013) a minimum of 15 positive and 30 negative samples were required.
Ethical considerations
The Research Ethical Committees of each institution approved the protocol for this study (Protocol no. 05714813.2.0000.5091). Written informed consent was obtained from all patients, healthy participants or legal guardians of all minors at the time of blood collection.
Serum samples
To determine the sensitivity, specificity and reproducibility of the DAT-LPC by above listed laboratories, serum samples recovered from the serum collection properly stored in freezer −80°C at laboratory 1 were used. To estimate the sensitivity, sera from 207 patients with confirmed clinical and parasitological diagnosis of VL were included. The specificity was evaluated using sera from 80 individuals: 19 patients with Chagas diseases, 18 patients with schistosomiasis (Schistosoma mansoni), 15 patients with tegumentary leishmaniasis and 28 healthy individuals.
The samples were blinded and aliquoted into four tubes each containing glycerol 1:1. Next, one aliquot of each sample was sent to the participant laboratories where they were tested with DAT-LPC prototype kit.
DAT-LPC prototype kit
The DAT-LPC prototype kit has been produced at CPqRR/FOCRUZ following the previously proposed modifications (Oliveira et al. Reference Oliveira, Saliba, Andrade and Rabello2011, Reference Oliveira, Saliba, Saliba and Rabello2013). Each prototype kit was composed of 10 antigen vials, one 10× concentrated rehydration solution vial (9% NaCl, 1% NaN3), one 10× concentrated sample diluent vial (NAC 5 mm, 9% NaCl, 1% NaN3) and instructions for use. The prototype kit is stable for 24 months and sufficient for quantitative assays of 80 samples or for qualitative assays of 460 samples. The resuspended antigen is stable in refrigerator (2–8°C) during two months.
Professional training and DAT-LPC procedure
An explanation addressing the study design was presented in each participant laboratory. The test procedure was demonstrated and two professionals of each laboratory performed the test at least in one sample, as follows:
First, the rehydration solution (10×) was diluted 1:10 with de-ionized water and 5 mL were added into the antigen vial. Next, the sera were diluted in sample diluents (previously diluted 1:10 with de-ionized water), and a 2-fold dilution series was made from 1:100 to 1:102 400. Then, 50 µL of DAT-LPC antigen suspension were added to each well of a V-shaped microtitre plate (Greiner Bio One Produtos Médicos Hospitalares Ltda, Americana, Brazil), containing 50 µL of diluted sera. After a minimal incubation of 4 h at room temperature in the dark, the end-titre was read as the dilution immediately before the well with a clear sharp-edged blue spot identical in size to the reaction negative control. Appropriate negative and positive control samples with known DAT titres were included. The cut-off adopted for interpretation of positive and negative results was 1:100, as previously defined (Oliveira et al. Reference Oliveira, Pedras, Assis and Rabello2009, Reference Oliveira, Saliba, Andrade and Rabello2011, Reference Oliveira, Saliba, Saliba and Rabello2013).
Statistical analysis
The DAT-LPC results obtained by each partner laboratory were decrypted and analysed at laboratory 1. The results obtained in this laboratory were taken as the reference for further comparisons with the results obtained by the other laboratories.
The sensitivity, specificity and diagnostic accuracy were calculated using two-by-two contingency table with exact binomial 95% CI using the Open-Epi Software (Dean et al. Reference Dean, Sullivan and Soe2015). Qui-square test was employed for comparison of sensitivities, specificities and diagnostic accuracies presented by the DAT-LPC performed in each partner laboratory, considering a 5% significance level (Bhattacharyya and Johnson, Reference Bhattacharyya and Johnson1977). The inter-laboratory reproducitibily of the qualitative DAT-LPC results was measured using Cohen's κ coefficient (Cohen, Reference Cohen1968) and interpreted following Landis and Koch (Reference Landis and Koch1977): values <0·2 were considered negligible; between 0·1 and 0·4 as weak; 0·41–0·6 as moderate; between 0·61 and 0·80 as good; and between 0·81 and 1·0 as excellent agreement.
To evaluate and represent the reproducibility of the quantitative results the end-titre were transformed into Log10 and individually assessed with the W-test for normality using GraphPad Prism 6.0 (GraphPad Software, Inc., La Jolla, CA, USA). One-hundred randomly selected results were stratified in ascending order in a Microsoft Office Excel 2007 spreadsheet. A graph was generated to define the area of one Log10[1titre] above and below of the end-titre results of laboratory 1 (reference laboratory). The end-titre results obtained in laboratories 2, 3 and 4 were overlapped with this area to verify how many were inside or matching the maximum or minimum limits of this area.
RESULTS
In laboratory 1, the DAT-LPC presented only one false-negative and two false-positive results. The performance in detecting VL positive samples was highest in laboratories 3 and 4, followed by laboratory 2. The DAT-LPC presented three false-positive results in laboratory 3 and two in laboratories 2 and 4.
Table 1 shows the sensitivities, specificities and diagnostic accuracy of the DAT-LPC found in all the participant laboratories. The highest sensitivity, specificity and diagnostic accuracy were observed in laboratory 1, which was the reference laboratory. In the other laboratories, the DAT-LPC sensitivities varied between 96·6 and 98·6%, the specificities ranged from 96·3 to 97·5%, and the diagnostic accuracies were between 96·9 and 98·3% (Table 1). Differences in results were statistically not significant (P > 0·05).
Sensitivity, specificity and diagnostic accuracy of DAT-LPC in four laboratories of different institutions
The data are presented as absolute () and relative % numbers with confidence interval of 95%.
Difference between sensitivities: Lab1 × Lab2: P = 0·07; Lab1 × Lab3: P = 0·60; Lab1 × Lab4: P = 0·60.
Difference between specificities: Lab1 × Lab3: P = 1·00.
Difference between diagnostic accuracies: Lab1 × Lab2: P = 0·08; Lab1 × Lab3: P = 0·70; Lab1 × Lab4: P = 0·50.
When assessed on two-by-two contingency table, six samples presented discordant results between laboratories 1 and 2, indicating lowest agreement of the qualitative DAT-LPC results (κ = 0·95). The highest agreement was found between laboratories 1 and 4, with only two discordant samples (κ = 0·98) (Table 2).
Inter-laboratories agreement of the qualitative DAT-LPC results, considering laboratory 1 as the reference
a Confidence interval with 95% (CI 95%).
Although all the laboratories received the same training regarding the test procedure and interpretation of the results, we observed a high number of discordant end-titre results. Laboratory 2 presented 31 concordant and 69 discordant end-titre results (agreement = 31%). A high number (23/69) of discordant end-titre results were observed in the zone with Log10[titre] varying from 4·11 to 4·71 (zone F). Laboratory 3 presented 78 concordant and 22 discordant end-titre results (agreement = 78%), the latter being more frequent (8/22) in the zone with Log10[titre] ranging 3·51–4·11, (zone D). Finally, 85 concordant and 15 discordant end-titre results were observed in laboratory 4, corresponding to an agreement of 85% (Fig. 1).
Reproducibility of 100 Log10[titre] of DAT-LPC end-titre results from VL patients, considering Lab1 as the reference. One hundred end-titre results from Lab 2 (A), Lab 3 (B) and Lab 4 (C) were transformed in Log10 and overlapped in an area produced by 100 Log10[titre ± 1] end-titre results obtained in Lab 1 (Reference Laboratory). Zone A: Log10[titre] from 2·0 to 2·31; Zone B: Log10[titre] from 2·9 to 3·51; Zone C: Log10[titre] from 3·2 to 3·81; Zone D: Log10[titre] from 3·5 to 4·11; Zone E: Log10[titre] from 3·8 to 4·41; Zone F: Log10[titre] from 4·1 to 4·71 and Zone G: Log10[titre] from 4·4 to 5·0.
DISCUSSION
Despite the technological advances observed in the last years concerning the diagnosis of infectious diseases, the DAT is still a good alternative for VL diagnosis. DAT is an easy-to-use, highly accurate and well-validated serological test that has been used for VL diagnosis in the last two decades (Srividya et al. Reference Srividya, Kulshrestha, Singh and Salotra2012).
Herein we evaluated the sensitivity, specificity, accuracy and reproducibility of the DAT-LPC by four different laboratories. The DAT-LPC sensitivities and specificities were statistically not different (P > 0·05) among the four laboratories and were similar to those found in the phase I study of the DAT-LPC, which revealed sensitivity of 99% and specificity of 98·2% (Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013).
The sensitivity and specificity results observed with DAT-LPC were higher than those previously described in another multi-centric study performed by four different public health institutions in Brazil (Machado de Assis et al. Reference Machado de Assis, Braga, Pedras, Oliveira, Barral, Siqueira, Costa, Costa, Holanda, Soares, Biá, Caldas, Romero and Rabello2011). The higher performance observed in the present study may be explained by the antigen preparation, freeze-dried in the present study and liquid in the previous one.
In the present work, the sample that presented the false-negative result in laboratory 1 (reference) was also negative in the other participant laboratories. Furthermore, the two samples (2/80) that presented false-positive results in laboratory 1 were also positive in the other participant laboratories. These samples were collected from patients with cutaneous leishmaniasis and were considered as false-positive VL results. This was expected as a small number (2/20) of false-positives serum samples from cutaneous leishmaniasis patients presenting lower titres were also found in the previous DAT-LPC study (Oliveira et al. Reference Oliveira, Saliba, Saliba and Rabello2013). Therefore, any positive results must be interpreted in light of other clinical, epidemiological and laboratory data, so as to achieve the correct diagnosis.
Our data suggest that not all of the participant professionals achieved proficiency in performing the DAT-LPC test. Discordant results were frequently found in laboratory 2, in which six samples presented false-negative results, despite of being from VL patients and having presented titres ranging from 1:800 to 1:12 800 defined by the reference laboratory (laboratory 1). Three discordant results were observed in Lab3, with two out of three samples with positive results in laboratory 1 showing negative results (Table 2). These discordant data, along with our observations that the participant professionals usually do not have problems with the execution of the test procedure, suggest that the professionals from laboratories 2 and 3 experienced difficulties in reading the results of the samples with lower titres.
The lowest number of discordant results (2/287) was observed in laboratory 4. This is probably because the participant professionals were benefiting from the experience gained previously when they integrated a study that evaluated the implementation and direct cost of the IT-LEISH® (BioRad Laboratories, Marnes-la-Coquette, FR) and the DAT-LPC in an urban area in Brazil (Machado de Assis et al. Reference Machado de Assis, Guimarães, Oliveira, Peruhype- Magalhães, Gomes and Rabello2015).
The discordant end-titre results were situated mainly in the zones 3·5–4·11 and 4·1–4·71 (zones D and F). The concordance in the end-titre readings have been a challenge for DAT users and it has been attributed to the subjective endpoint reading of the results (Adams et al. Reference Adams, Jacquet, Schoone, Gidwani, Boelaert and Cunningham2012). Despite having received instructions to record end-titre results, it is likely that some professionals established end-titre results with 50% of agglutination of the well, while other professionals recorded end-titre results as the dilution immediately before the well with a clear sharp-edged blue spot identical to the negative control. A similar problem has been reported in a multicentre study by Adams et al. (Reference Adams, Jacquet, Schoone, Gidwani, Boelaert and Cunningham2012), in which, at first, 50% of discordant DAT end-titre results were found among the laboratories involved. Posterior analyses of photographs of the DAT plates, however, indicated that part of the discordant end-titre results were due to divergent interpretation of the results. The authors pointed out that photodocumentation is a solution to increase the usefulness of DAT (Adams et al. Reference Adams, Jacquet, Schoone, Gidwani, Boelaert and Cunningham2012). Thus, the reproducibility of end-titre results of our DAT-LPC test could probably be increased following the implementation of more consistent analyses and interpretation of the data, which must be performed in the future when the kit is produced and commercialized by a Brazilian manufacturer. A partnership public–private was established and the scale-up of the DAT-LPC production is under way.
In summary, we evaluated the use of DAT-LPC for the diagnosis of VL and it demonstrated excellent reproducibility and diagnostic accuracy in all public health participating laboratories involved. The test proved to be accurate for the diagnosis of human VL in a research laboratory (laboratory 1), a public health reference laboratory (laboratory 2), an infectious diseases reference hospital (laboratory 3), and a municipal public laboratory (Lab4). Although blinded samples were used, it is important to note that the present is a case control validation study, which may justify the existence of an over estimated reproducitibily and diagnostic accuracy. This is a pilot study demonstrating the usefulness of the DAT-LPC as a confirmatory VL test in different scenarios and larger field trials will be needed to confirm the suitability of replacing IFAT with DAT-LPC for VL diagnosis in the Brazilian public health system.
ACKNOWLEDGEMENTS
We thank to Ezequiel Dias Foundation, Eduardo de Menezes Hospital and Health Secretary of Ribeirão das Neves for support necessary to perform this work. We also thank Marcelo Antônio Pascoal Xavier for the help with the statistical analysis.
FINANCIAL SUPPORT
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
