BeckerGuides: Systematic Review Guide: Risk of Bias Tools

Risk of Bias

Each article that has passed the screening process for a systematic review should be assessed for risk of bias. The Cochrane manual defines risk of bias as "a systematic error, or deviation from the truth, in results." It further states that "Biases can lead to under-estimation or over-estimation of the true intervention effect and can vary in magnitude: some are small (and trivial compared with the observed effect) and some are substantial (so that an apparent finding may be due entirely to bias)." (Cochrane Manual: https://training.cochrane.org/handbook/current/chapter-07)

The resources listed below can aid in determining risk of bias scores for studies included in your systematic review.

Risk of Bias Resources

Quality Assessment and Risk of Bias Tool Repository
A resource for finding and selecting a risk of bias or quality assessment tool for evidence synthesis projects. This project was created and is maintained by Duke University's Medical Center Library & Archives. You can search this resource by study design type or name of tool. It is a good resource to use if you are unsure which risk of bias tool will work best for your project.
Risk of Bias Visualization Tool (ROBVIS)
Robvis is a web app designed to for visualizing risk-of-bias assessments performed as part of a systematic review. The tool creates: “traffic light” plots of the domain-level judgements for each individual result; and weighted bar plots of the distribution of risk-of-bias judgements within each bias domain. The figures are of publication quality, and are formatted according the risk-of-bias assessment tool used to perform the assessments (e.g. RoB 2, ROBINS-I, or QUADAS-2).

Popular Risk of Bias Assessment Tools

Modified Downs and Black Checklist
Provides both an overall score for study quality and a numeric score out of a possible 30 points. It has five sections. Administration of the tool can happen either within a systematic review process, or as a quality assessment tool for individual articles.
Newcastle-Ottawa Quality Assessment Scale
A tool used to assess the quality of non-randomized studies. Allocates a maximum of nine stars, for quality of selection, comparability, exposure and outcome of study participants.
Cochrane Risk of Bias Tool (RoB 2)
Tool used to assess risk of bias for randomized controlled trials. Required for all Cochrane group systematic reviews.
Cochrane Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I Tool)
The ROBINS-I is a tool developed to assess risk of bias in the results of non-randomized studies that compare health effects of two or more interventions.
MMAT (Mixed Methods Appraisal Tool)
The MMAT is intended to be used as a checklist for concomitantly appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods studies)