https://orcid.org/0000-0003-1313-7922
Figures
Abstract
The soluble urokinase plasminogen activator receptor (suPAR) has been proposed as a biomarker for risk stratification of patients presenting with acute infections. However, most studies evaluating suPAR have used platform-based assays, the accuracy of which may differ from point-of-care tests capable of informing timely triage in settings without established laboratory capacity. Using samples and data collected during a prospective cohort study of 425 patients presenting with moderate Covid-19 to two hospitals in India, we evaluated the analytical performance and prognostic accuracy of a commercially-available rapid diagnostic test (RDT) for suPAR, using an enzyme-linked immunosorbent assay (ELISA) as the reference standard. Our hypothesis was that the suPAR RDT might be useful for triage of patients presenting with moderate Covid-19 irrespective of its analytical performance when compared with the reference test. Although agreement between the two tests was limited (bias = -2.46 ng/mL [95% CI = -2.65 to -2.27 ng/mL]), prognostic accuracy to predict supplemental oxygen requirement was comparable, whether suPAR was used alone (area under the receiver operating characteristic curve [AUC] of RDT = 0.73 [95% CI = 0.68 to 0.79] vs. AUC of ELISA = 0.70 [95% CI = 0.63 to 0.76]; p = 0.12) or as part of a published multivariable prediction model (AUC of RDT-based model = 0.74 [95% CI = 0.66 to 0.83] vs. AUC of ELISA-based model = 0.72 [95% CI = 0.64 to 0.81]; p = 0.78). Lack of agreement between the RDT and ELISA in our cohort warrants further investigation and highlights the importance of assessing candidate point-of-care tests to ensure management algorithms reflect the assay that will ultimately be used to inform patient care. Availability of a quantitative point-of-care test for suPAR opens the door to suPAR-guided risk stratification of patients with Covid-19 and other acute infections in settings with limited laboratory capacity.
Citation: Chandna A, Mahajan R, Gautam P, Mwandigha L, Dittrich S, Kumar V, et al. (2023) Point-of-care prognostication in moderate Covid-19: Analytical validation and prognostic accuracy of a soluble urokinase plasminogen activator receptor (suPAR) rapid test. PLOS Glob Public Health 3(8): e0001538. https://doi.org/10.1371/journal.pgph.0001538
Editor: Krutika Kuppalli, World Health Organization, SWITZERLAND
Received: December 1, 2022; Accepted: July 20, 2023; Published: August 21, 2023
Copyright: © 2023 Chandna et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: De-identified, individual participant data from this study will be available to researchers whose proposed purpose of use is approved by the data access committees at Médecins Sans Frontières and the Mahidol-Oxford Tropical Medicine Research Unit. Inquiries or requests for the data may be sent to data.sharing@london.msf.org and datasharing@tropmedres.ac. Researchers interested in accessing biobanked samples should contact the corresponding authors who will coordinate with the respective institutions.
Funding: The PRIORITISE (Prognostication of Oxygen Requirement in Patients with Non-severe SARS-CoV-2 Infection) study was funded by Médecins Sans Frontières, India. The Wellcome Trust provides core funding to the Mahidol-Oxford Tropical Medicine Research Unit in Bangkok [220211 to MORU; 215604/Z/19/Z to YL], which supported the design, monitoring and analysis of the study. The suPAR RDTs were procured by FIND with funding from the Australian Government. CK is supported by a Wellcome Trust/Royal Society Sir Henry Dale Fellowship [211182/Z/18/Z]. Médecins Sans Frontières maintained a sponsor/investigator role for the study. All other funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. For the purpose of open access, the author has applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: Sabine Dittrich and Jennifer Osborn declare that they are employed by FIND. All other authors have declared that no competing interests exist.
Introduction
In busy clinical settings the window for effective triage is short. Biochemical biomarkers included in risk stratification tools require rapid turnaround times. However, studies developing triage tools commonly use laboratory-based platforms to quantify biomarker concentrations, the accuracy of which may differ from point-of-care tests required to inform timely management of individual patients in settings with limited laboratory capacity [1].
The soluble version of the urokinase plasminogen activator receptor (suPAR) is upregulated during the host response to infection [2]. Measurements of suPAR have been shown to be useful in the diagnosis and prognosis of a wide range of infections and infectious syndromes [3–8], including SARS-CoV-2 [9,10]. Recently, our group and others developed clinical prediction models incorporating suPAR for both community- and hospital-based triage of patients with Covid-19 [11,12]. However, although suPAR is measurable using a commercially-available rapid test, these studies quantified suPAR using laboratory-based immunoassays.
We performed an analytical validation of a rapid diagnostic test (RDT) for suPAR using samples from a multi-centre prospective cohort study conducted in India [11]. We evaluated the prognostic accuracy of the RDT by comparing the predictive performance of suPAR quantified using the RDT and an enzyme-linked immunosorbent assay (ELISA). Our hypothesis was that the suPAR RDT might be useful for triage of patients presenting with moderate Covid-19 irrespective of its analytical performance when compared with the reference test.
Methods
Ethics statement
Ethical approval was given by the All India Institute for Medical Sciences Patna Ethics Committee; Christian Medical College Ethics Committee; Oxford Tropical Research Ethics Committee; and Médecins Sans Frontières Ethical Review Board. All participants provided informed written consent prior to recruitment.
Study population and clinical data collection
This is a secondary analysis of data collected during a prospective cohort study conducted at two hospitals in India between October 2020 and July 2021. The study design and setting have been described previously [11]. Briefly, consenting consecutive adults (aged ≥ 18 years) presenting with clinically-suspected SARS-CoV-2 infection of moderate severity (defined as peripheral oxygen saturation (SpO2) ≥ 94% and respiratory rate (RR) ≤ 30 breaths per minute (BPM) in the context of systemic symptoms) were recruited. Clinical parameters were measured at enrolment. Admitted participants were followed-up each day until death, discharge or day 14, whichever occurred first. For those not admitted or discharged prior to day 14, follow-up was conducted via telephone on days 7 and 14, with participants who reported worsening and/or persistent symptoms recalled to have their SpO2 and RR measured. The primary outcome for the original study was development of a supplemental oxygen requirement within 14 days of enrolment, defined as any of: SpO2 < 94%; RR > 30 BPM; SpO2/FiO2 < 400; or death.
Laboratory procedures
Venous blood samples were collected in ethylenediaminetetraacetic acid (EDTA) tubes at enrolment, centrifuged within four hours, and plasma aliquots stored at ≤ -20°C. Frozen plasma aliquots were transported on dry ice and thawed at 2-8°C overnight prior to analysis. suPAR concentrations were quantified using the suPARnostic ELISA and the suPARnostic Quick Triage test (Virogates, Denmark). The suPARnostic ELISA is a simplified double monoclonal antibody sandwich assay which requires 15μL of plasma. Samples were run in duplicate and mean concentration reported. The Quick Triage test is a RDT based on lateral flow principles. It requires 10μL of plasma and has a dynamic range of 2–15 ng/mL. Paired with an automated lateral flow optical reader it has a time-to-result of 20 minutes. Both tests were performed as per the manufacturer’s instructions using the same aliquot of thawed plasma [13,14], and the operators who performed both tests were blinded to the results of the other test.
Primary and secondary outcomes
The primary outcome was the prognostic accuracy of the RDT (index test) assessed by comparing the predictive performance (area under the receiver operating characteristic curve; AUC) of the RDT to the ELISA (reference standard). The secondary outcome was the analytical performance of the RDT, assessed by quantifying the agreement between the RDT and ELISA.
Statistical methods
Logistic regression was used to quantify the AUC and compare (DeLong method) the prognostic accuracy of the RDT and ELISA (R package: pROC) [15,16]. The analytical performance of the RDT was evaluated using a Bland-Altman plot to estimate the bias and limits of agreement between the RDT and the ELISA (R package: blandr) [17,18]. Assessment of agreement was limited to samples within the dynamic range (2–15 ng/mL) of the RDT. A sensitivity analysis was conducted where samples quantified on the ELISA but outside the dynamic range of the RDT were set to the limits of quantification of the RDT. All analyses were performed in R, versions 4.0.2 and 4.0.3 [19].
Sample size
For the purposes of an analytical validation, the Clinical and Laboratory Standards Institute (CLSI) recommend a minimum sample size of 100 to evaluate agreement between a candidate and reference test [20]. To maximise precision of the results, we used all available samples from the original study (n = 425).
Study reporting
This investigator-initiated study was prospectively registered (ClinicalTrials.gov; NCT04441372), with protocol and statistical analysis plan uploaded to the Open Science Framework platform (DOI: 10.17605/OSF.IO/DXQ43). The study is reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) guidelines (Table A in S1 Text) [21].
Results
Clinical outcomes
Between 22 October 2020 and 3 July 2021, 426 participants were recruited of whom 425 had suPAR concentrations quantified using the RDT and ELISA. Three participants were lost-to-follow-up before day 14, leaving 422 participants available for the primary analysis (Fig 1). Eighty-nine participants developed a supplemental oxygen requirement (89/422; 21.1%).
Limited agreement between the reference ELISA and candidate RDT for quantification of suPAR
Forty-four samples returned values outside the dynamic range of the RDT on either the ELISA and/or RDT, leaving 381 paired samples for assessment of agreement. Median suPAR concentration was higher when quantified by the RDT (Table 1; 6.6 vs. 4.2 ng/mL; p < 0.001). The two tests were correlated (Pearson’s correlation = 0.66 [95% CI = 0.60–0.71]; p < 0.001) but there was limited agreement, with the RDT returning higher values than the ELISA on average. A Bland-Altman plot indicated a bias of -2.46 ng/mL (95% CI = -2.65 to -2.27 ng/mL) with upper and lower limits of agreement of 1.21 ng/mL (95% CI = 0.89 to 1.54 ng/mL) and -6.13 ng/mL (95% CI = -6.45 to -5.81 ng/mL) respectively (Fig 2). A sensitivity analysis in which samples outside the dynamic range of the RDT were set to the limits of detection of the RDT returned similar results (Fig A in S1 Text). Given the disagreement with the reference test we verified the reproducibility of the RDT results by repeating the measurements of all participants at one site (n = 125) using another batch of RDTs (Fig B in S1 Text).
Left panel: Bland-Altman plot indicating agreement between the two tests. Difference between RDT and ELISA measurement in ng/mL plotted on y-axis. Mean of the RDT and ELISA measurement in ng/mL plotted on x-axis. Limits of agreement defined by the concentrations within which 95% of the data fall. Blue line indicates bias, green line indicates upper limit of agreement, red line indicates lower limit of agreement, all with 95% confidence intervals. Right panel: Scatterplot indicating correlation between the two tests.
Median values (IQR) are reported for continuous variables. *3 participants missing information about supplemental oxygen requirement excluded from table but included in assessment of agreement. †Wilcoxon rank sum test.
Prognostic accuracy is maintained when a rapid point-of-care test is used to quantify suPAR concentrations instead of a laboratory-based assay
Although agreement between the two tests was limited, we recognised that the RDT might still have utility providing predictive performance did not deteriorate when suPAR concentrations were quantified using the RDT rather than the ELISA. Participants who progressed to develop a supplemental oxygen requirement had higher median baseline suPAR levels, irrespective of the assay used for quantification (Table 1; RDT = 8.0 vs. 6.2 ng/mL, p < 0.001; ELISA = 5.2 vs. 4.0 ng/mL, p < 0.001). Prognostic accuracy of the RDT was comparable to the ELISA (Fig 3; AUC of RDT = 0.73 [95% CI = 0.68 to 0.79] vs. AUC of ELISA = 0.70 [95% CI = 0.63 to 0.76]; p = 0.12) for discriminating participants who would progress to require supplemental oxygen. Evaluation of the prognostic accuracy of the RDT within the framework of our previously published multivariable clinical prediction model confirmed comparable predictive performance of the model whether suPAR was quantified using the RDT or the ELISA (AUC of RDT-based model = 0.74 [95% CI = 0.66 to 0.83] vs. AUC of ELISA-based model = 0.72 [95% CI = 0.64 to 0.81]; p = 0.78). The weighting (regression coefficient) for suPAR within the model differed depending on which assay was used for quantification (Text A in S1 Text).
AUC = area under the receiver operating characteristic curve.
Discussion
We demonstrate equivalent prognostic accuracy between an index RDT for suPAR and a laboratory-based ELISA to predict progression to supplemental oxygen requirement amongst patients presenting with moderate Covid-19. Comparable predictive performance of the RDT was achieved despite limited agreement with the reference test and was maintained whether suPAR was used alone or as a constituent parameter in a previously published multivariable clinical prediction model for the triage of patients with moderate Covid-19 [11].
Our results illustrate the importance of conducting both analytical and clinical validations of candidate point-of-care tests. The limited agreement between the two suPAR tests indicates that the RDT cannot replace or be used interchangeably with the ELISA, a conclusion that is likely to remain valid for other biomarkers and assays unless strong agreement has been demonstrated. Consequently, results from studies utilising different suPAR assays should not be pooled. If the tests are adopted for routine use, cut-offs associated with particular clinical management decisions (for example, admission or discharge from the emergency department) or weightings within multivariable triage tools (prediction models) should be assay-dependent [11,12,22]. Similarly, if suPAR measurements are used to inform participant recruitment into clinical trials, it is important that eligibility criteria are tailored to the assay used for enrolment [23].
The lack of agreement between the RDT and ELISA in our cohort is unexpected. The manufacturer reports that suPAR concentrations measured using the RDT should be within ±10% of measurements made on the ELISA, further underlining the need for context specific evaluation [13,22]. The impact of different detection methods for suPAR has been demonstrated [24], however in our study both the ELISA and RDT used the same capture antibodies. Although RDT measurements were made after an additional freeze-thaw cycle, multiple studies have confirmed that suPAR concentrations are stable up to at least five repeated freeze-thaw cycles [25,26]. In our cohort, suPAR concentrations quantified using the RDT were higher than anticipated for non-severe patients attending an emergency department [13]. It is possible that an unknown factor may have interfered with the functioning of the RDT assay in this population and this merits further investigation. To our knowledge only one other study has quantified suPAR concentrations using the same RDT in India; 66.3% (126/190) of patients had suPAR concentrations > 5.5 ng/mL, however average concentration was not reported [27]. Nevertheless, as our results demonstrate, lack of agreement between inexpensive, quick, and practical RDTs and batched, quality-controlled, laboratory-based assays, does not necessarily preclude clinical utility of an index test on the field.
This is the first study evaluating the analytical performance and prognostic accuracy of a RDT for suPAR, head-to-head against a reference test in resource-constrained setting. The RDTs were performed on frozen plasma by laboratory technicians, rather than on fresh plasma as suggested by the manufacturer. If the tests were to inform real-time clinical decisions, fresh plasma would be used and trained laboratory technicians may not be available, especially in contexts with limited laboratory capacity. Future research should extend our results to explore the field-based implementation of the RDT using unfrozen patient samples and evaluate reliability and usability amongst lesser-trained practitioners.
We demonstrate comparable prognostic accuracy of a rapid, quantitative, point-of-care test for suPAR to the reference laboratory-based ELISA that was used to develop a previously validated clinical prediction model for the triage of patients presenting with moderate Covid-19. These results are promising and should encourage further exploration of the utility of suPAR-guided risk stratification of patients presenting with Covid-19 and other acute infections in settings with limited laboratory capacity. Lack of agreement between the two tests highlights the importance of undertaking evaluations of point-of-care tests to ensure cut-offs and weightings are adjusted accordingly prior to a test being recommended for clinical use.
Supporting information
S1 Text. Supplementary appendix.
Table A: STARD checklist. Fig A: Agreement between the suPAR RDT and ELISA. Fig B: Agreement between repeated RDT measurements. Text A: Prognostic accuracy of the suPAR RDT within the framework of a clinical prediction model.
https://doi.org/10.1371/journal.pgph.0001538.s001
(DOCX)
Acknowledgments
The authors thank Emmanuel Moreau at the Foundation for Innovative New Diagnostics (FIND) for his advice during the conduct of the study.
References
- 1. Dittrich S, Tadesse BT, Moussy F, Chua A, Zorzet A, Tangden T, et al. Target Product Profile for a Diagnostic Assay to Differentiate between Bacterial and Non-Bacterial Infections and Reduce Antimicrobial Overuse in Resource-Limited Settings: An Expert Consensus. PLoS One. 2016;11(8):e0161721. Epub 2016/08/26. pmid:27559728; PubMed Central PMCID: PMC4999186.
- 2. Donadello K, Scolletta S, Covajes C, Vincent JL. suPAR as a prognostic biomarker in sepsis. BMC Med. 2012;10.
- 3. Ni W, Han Y, Zhao J, Cui J, Wang K, Wang R, et al. Serum soluble urokinase-type plasminogen activator receptor as a biological marker of bacterial infection in adults: a systematic review and meta-analysis. Sci Rep. 2016;6:39481. Epub 2016/12/20. pmid:27991579; PubMed Central PMCID: PMC5172370.
- 4. Wittenhagen P, Kronborg G, Weis N, Nielsen H, Obel N, Pedersen SS, et al. The plasma level of soluble urokinase receptor is elevated in patients with Streptococcus pneumoniae bacteraemia and predicts mortality. Clin Microbiol Infect. 2004;10(5):409–15. Epub 2004/04/29. pmid:15113317.
- 5. Rabna P, Andersen A, Wejse C, Oliveira I, Gomes VF, Haaland MB, et al. Utility of the plasma level of suPAR in monitoring risk of mortality during TB treatment. PLoS One. 2012;7(8):e43933. Epub 2012/09/01. pmid:22937128; PubMed Central PMCID: PMC3429420.
- 6. Stefanova V, Ngai M, Weckman AM, Wright JK, Zhong K, Richard-Greenblatt M, et al. suPAR as a Prognostic Marker of Ugandan Children at Risk of Severe and Fatal Malaria. Clin Infect Dis. 2022. Epub 2022/06/09. pmid:35675322.
- 7. Luo Q, Ning P, Zheng Y, Shang Y, Zhou B, Gao Z. Serum suPAR and syndecan-4 levels predict severity of community-acquired pneumonia: a prospective, multi-centre study. Crit Care. 2018;22(1):15. Epub 2018/01/26. pmid:29368632; PubMed Central PMCID: PMC5784729.
- 8. Pregernig A, Muller M, Held U, Beck-Schimmer B. Prediction of mortality in adult patients with sepsis using six biomarkers: a systematic review and meta-analysis. Ann Intensive Care. 2019;9(1):125. Epub 2019/11/11. pmid:31705327; PubMed Central PMCID: PMC6841861.
- 9. Arnold DT, Attwood M, Barratt S, Morley A, Elvers KT, McKernon J, et al. Predicting outcomes of COVID-19 from admission biomarkers: a prospective UK cohort study. Emerg Med J. 2021;38(7):543–8. Epub 2021/05/23. pmid:34021028; PubMed Central PMCID: PMC8206177.
- 10. Rovina N, Akinosoglou K, Eugen-Olsen J, Hayek S, Reiser J, Giamarellos-Bourboulis EJ. Soluble urokinase plasminogen activator receptor (suPAR) as an early predictor of severe respiratory failure in patients with COVID-19 pneumonia. Crit Care. 2020;24(1):187. Epub 2020/05/02. pmid:32354367; PubMed Central PMCID: PMC7191969.
- 11. Chandna A, Mahajan R, Gautam P, Mwandigha L, Gunasekaran K, Bhusan D, et al. Facilitating safe discharge through predicting disease progression in moderate COVID-19: a prospective cohort study to develop and validate a clinical prediction model in resource-limited settings. Clin Infect Dis. 2022. Epub 2022/03/25. pmid:35323932.
- 12. Chalkias A, Skoulakis A, Papagiannakis N, Laou E, Tourlakopoulos K, Pagonis A, et al. Circulating suPAR associates with severity and in-hospital progression of COVID-19. Eur J Clin Invest. 2022;52(7):e13794. Epub 2022/04/19. pmid:35435245; PubMed Central PMCID: PMC9111634.
- 13. Virogates A/S Denmark. Instructions for use: suPARnostic Quick Triage for aLF Reader. Birkerød, Denmark 2020.
- 14. Virogates A/S Denmark. Instructions for use: Enzyme immunoassay for quantitative determination of soluble urokinase plasminogen activator receptor in human plasma and serum. Birkerød, Denmark 2020.
- 15. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12(77). pmid:21414208
- 16. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988;44:837–45. pmid:3203132
- 17. Bland M, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327(8476):307–10. pmid:2868172
- 18. Datta D. blandr: a Bland-Altman Method Comparison package for R. 2017.
- 19.
R Core Team. R: A language and environment for statistical computing. VIenna, Austria: R Foundation for Statistical Computing; 2020.
- 20. Clinical and Laboratory Standards Institute. Measurement Procedure Comparison and Bias Estimation Using Patient Samples; Approved Guideline—Third Edition (EP09-A3). Philadelphia, USA: 2013 Contract No.: 11.
- 21. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. Epub 2015/10/30. pmid:26511519; PubMed Central PMCID: PMC4623764.
- 22. Stauning MA, Altintas I, Kallemose T, Eugen-Olsen J, Lindstrom MB, Rasmussen LJH, et al. Soluble Urokinase Plasminogen Activator Receptor as a Decision Marker for Early Discharge of Patients with COVID-19 Symptoms in the Emergency Department. J Emerg Med. 2021;61(3):298–313. Epub 2021/06/08. pmid:34092446; PubMed Central PMCID: PMC7997406.
- 23. Kyriazopoulou E, Poulakou G, Milionis H, Metallidis S, Adamis G, Tsiakos K, et al. Early treatment of COVID-19 with anakinra guided by soluble urokinase plasminogen receptor plasma levels: a double-blind, randomized controlled phase 3 trial. Nat Med. 2021;27(10):1752–60. Epub 2021/09/05. pmid:34480127; PubMed Central PMCID: PMC8516650.
- 24. Winnicki W, Sunder-Plassmann G, Sengolge G, Handisurya A, Herkner H, Kornauth C, et al. Diagnostic and Prognostic Value of Soluble Urokinase-type Plasminogen Activator Receptor (suPAR) in Focal Segmental Glomerulosclerosis and Impact of Detection Method. Sci Rep. 2019;9(1):13783. Epub 2019/09/26. pmid:31551522; PubMed Central PMCID: PMC6760112.
- 25. Chew-Harris J, Appleby S, Richards AM, Troughton RW, Pemberton CJ. Analytical, biochemical and clearance considerations of soluble urokinase plasminogen activator receptor (suPAR) in healthy individuals. Clin Biochem. 2019;69:36–44. Epub 2019/05/28. pmid:31129182.
- 26. Kofoed K, Schneider UV, Scheel T, Andersen O, Eugen-Olsen J. Development and validation of a multiplex add-on assay for sepsis biomarkers using xMAP technology. Clin Chem. 2006;52(7):1284–93. Epub 2006/05/13. pmid:16690735.
- 27. Kumar P, Kakar A, Gogia A, Waziri N. Evaluation of soluble urokinase-type plasminogen activator receptor (suPAR) quick test for triage in the emergency department. J Family Med Prim Care. 2019;8(12):3871–5. Epub 2019/12/28. pmid:31879628; PubMed Central PMCID: PMC6924241.