Background: We critically evaluated the risk of bias in published systematic reviews
(SRs) and meta-analyses (MAs) pertaining to COVID-19 using ROBIS tool.
Materials And Methods: MEDLINE and Cochrane Central Library were searched for SRs/MAs on
14th May 2020, including studies of all designs describing various facets of COVID-19 in humans;
no restrictions were applied for interventions, comparators, and outcomes. Two reviewers
independently assessed all the SRs/MAs with ROBIS.
Results: Out of 204 identified records, 48 SRs/MAs were included. The most frequently reviewed
topics were therapy outcomes, diagnosis, and comorbidities (15, 8, and 6 papers respectively).
Only 29/48(60.41%) papers had made a mention of using PRISMA or other guidelines for drafting
the SR/MA. Only 5/48(10.42%) of all included SRs/MAs had low overall risk of bias as per
ROBIS tool; 41/48(85.42%) had high risk of bias, 2/48(4.17%) had unclear risk of bias. The
highest proportion of bias was found in data synthesis and findings (30/48, 62.50% of studies had
high risk of bias), followed by study identification and selection (29/48, 60.42%). The IRR for
methodological quality assessment was substantial, with the Cohen’s kappa values being 0.64,
0.68, 0.62, and 0.75 for domains 1-4 of ROBIS tool, and 0.66 for overall risk of bias assessment.
Conclusion: There are serious concerns about the methodology employed to generate the results of
the SRs/ MAs pertaining to COVID-19, with ‘quantity’ seemingly being given more importance
than ‘quality’ of the paper.