Skip to main content
Becker Medical Library logotype
Library Quicklinks and Information

Systematic Reviews: Appraisal and Grading

Resources to Support Work in Systematic Reviews

Appraise the Literature

Secrets to Healthy Skepticism by,

Centre for Evidence-Based Medicine Worksheets

Duke Critical Appraisal Worksheets

Evaluation of studies of treatment or prevention interventions by, Nicky Cullum, RN, PhD

A Simple Method for Evaluating the Clinical Literature by,

Robert J. Flaherty , MD  
Uses PP-ICONS (Problem, Patient/Population, Intervention, Comparison, Number of Patients, Statistics).

JAMA's User’s Guide to the Medical Literature

BMJ's How To Read a Paper by, Trisha Greenhalgh

How to Read a Scientific Paper by, John W. Little and Roy Parker 

Articles on Grading

Please Note: Where possible I have provided free links to articles available for free, otherwise these are publications held by Becker Library and you will need to be on a network line or logged into proxy or VPN to access. 

  1. Ahmed, F., Temte, J. L., Campos-Outcalt, D., Schunemann, H. J., & for the ACIPEvidence Based Recommendations Work Group (EBRWG). (2011). Methods for developing evidence-based recommendations by the advisory committee on immunization practices (ACIP) of the U.S. centers for disease control and prevention (CDC). Vaccine, doi:10.1016/j.vaccine.2011.08.005
  2. American Academy of Pediatrics Steering Committee on Quality Improvement and Management. (2004). Classifying recommendations for clinical practice guidelines. Pediatrics, 114(3), 874-877. doi:10.1542/peds.2004-1260
  3. Ansari, M. T., Tsertsvadze, A., & Moher, D. (2009). Grading quality of evidence and strength of recommendations: A perspective. PLoS Medicine, 6(9), e1000151. doi:10.1371/journal.pmed.1000151
  4. Armola, R. R., Bourgault, A. M., Halm, M. A., Board, R. M., Bucher, L., Harrington, L., . . . 2008-2009 Evidence-Based Practice Resource Work Group of the American Association of Critical-Care Nurses. (2009). Upgrading the american association of critical-care nurses' evidence-leveling hierarchy. American Journal of Critical Care : An Official Publication, American Association of Critical-Care Nurses, 18(5), 405-409. doi:10.4037/ajcc2009170
  5. Atkins, D., Briss, P. A., Eccles, M., Flottorp, S., Guyatt, G. H., Harbour, R. T., . . . GRADE Working Group. (2005). Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system. BMC Health Services Research, 5(1), 25. doi:10.1186/1472-6963-5-25
  6. Atkins, D., Eccles, M., Flottorp, S., Guyatt, G. H., Henry, D., Hill, S., . . . GRADE Working Group. (2004). Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches the GRADE working group. BMC Health Services Research, 4(1), 38. doi:10.1186/1472-6963-4-38
  7. Balshem, H., Helfand, M., Schunemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., . . . Guyatt, G. H. (2011). GRADE guidelines: 3. rating the quality ofevidence. Journal of Clinical Epidemiology, 64(4), 401-406. doi:10.1016/j.jclinepi.2010.07.015
  8. Brozek, J. L., Akl, E. A., Alonso-Coello, P., Lang, D., Jaeschke, R., Williams, J. W., . . . GRADE Working Group. (2009). Grading quality of evidence and strength of recommendations in clinical practice guidelines. part 1 of 3. an overview of the GRADE approach and grading quality of evidence about interventions. Allergy, 64(5), 669-677. doi:10.1111/j.1398-9995.2009.01973.x
  9. Brozek, J. L., Akl, E. A., Compalati, E., Kreis, J., Terracciano, L., Fiocchi, A., . . . GRADE Working Group. (2011). Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3. the GRADE approach to developing recommendations. Allergy, 66(5), 588-595. doi:10.1111/j.1398-9995.2010.02530.x; 10.1111/j.1398-9995.2010.02530.x
  10. Canfield, S. E., & Dahm, P. (2011). Rating the quality of evidence and the strength of recommendations using GRADE. World Journal of Urology, 29(3), 311-317. doi:10.1007/s00345-011-0667-2
  11. Chong, C. A., Chen, I. J., Naglie, G., & Krahn, M. D. (2009). How well do guidelines incorporate evidence on patient preferences? Journal of General Internal Medicine, 24(8), 977-982. doi:10.1007/s11606-009-0987-8
  12. Coyne, D. W. (2007). Practice recommendations based on low, very low, and missing evidence. Clinical Journal of the American Society of Nephrology : CJASN, 2(1), 11-12. doi:10.2215/CJN.02980906
  13. Dykes, P. C. (2005). Translating evidence into practice for providers and patients. Evidence-Based Cardiovascular Medicine, 9(3), 157-159. doi:10.1016/j.ebcm.2005.06.049
  14. Ebell, M. H., Siwek, J., Weiss, B. D., Woolf, S. H., Susman, J., Ewigman, B., & Bowman, M. (2004). Strength of recommendation taxonomy (SORT): A patient-centered approach to grading evidence in the medical literature. The Journal of the American Board of Family Practice / American Board of Family Practice, 17(1), 59-67.
  15. Ebell, M. H., Siwek, J., Weiss, B. D., Woolf, S. H., Susman, J., Ewigman, B., & Bowman, M. (2004). Strength of recommendation taxonomy (SORT): A patient-centered approach to grading evidence in the medical literature. American Family Physician, 69(3), 548-556.
  16. Ebell, M. H., Siwek, J., Weiss, B. D., Woolf, S. H., Susman, J. L., Ewigman, B., & Bowman, M. (2004). Simplifying the language of evidence to improve patient care: Strength of recommendation taxonomy (SORT): A patient-centered approach to grading evidence in medical literature. The Journal of Family Practice, 53(2), 111-120.
  17. Fontanesi, J., Mendoza, S., Bowers, D., & Reznik, V. (2009). Translating operational research to the medical community: Using "guiding measurements" to improve the quality of healthcare delivery. The Journal of Medical Practice Management : MPM, 24(4), 248-253.
  18. Guyatt, G., Gutterman, D., Baumann, M. H., Addrizzo-Harris, D., Hylek, E. M., Phillips, B., . . . Schunemann, H. (2006). Grading strength of recommendations and quality of evidence in clinical guidelines: Report from an american college of chest physicians task force. Chest, 129(1), 174-181. doi:10.1378/chest.129.1.174
  19. Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., . . . Schunemann, H. J. (2011). GRADE guidelines: 1. introduction-GRADE evidenceprofiles and summary of findings tables. Journal of Clinical Epidemiology, 64(4), 383-394. doi:10.1016/j.jclinepi.2010.04.026
  20. Guyatt, G., Vist, G., Falck-Ytter, Y., Kunz, R., Magrini, N., & Schunemann, H. (2006). An emerging consensus on grading recommendations? Evidence-Based Medicine, 11(1), 2-4.
  21. Guyatt, G., Vist, G., Falck-Ytter, Y., Kunz, R., Magrini, N., & Schunemann, H. (2006). An emerging consensus on grading recommendations? ACP Journal Club,144(1), A8-9.
  22. Guyatt, G. H., Oxman, A. D., Kunz, R., Atkins, D., Brozek, J., Vist, G., . . . Schunemann, H. J. (2011). GRADE guidelines: 2. framing the question and deciding on important outcomes. Journal of Clinical Epidemiology, 64(4), 395-400. doi:10.1016/j.jclinepi.2010.09.012
  23. Guyatt, G. H., Oxman, A. D., Schunemann, H. J., Tugwell, P., & Knottnerus, A. (2011). GRADE guidelines: A new series of articles in the journal of clinical epidemiology. Journal of Clinical Epidemiology, 64(4), 380-382. doi:10.1016/j.jclinepi.2010.09.011
  24. Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., . . . GRADE Working Group. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.), 336(7650), 924-926. doi:10.1136/bmj.39489.470347.AD
  25. Harbour, R., & Miller, J. (2001). A new system for grading recommendations inevidence based guidelines. BMJ (Clinical Research Ed.), 323(7308), 334-336.
  26. Harris, C., Turner, T., Mazza, D., Wilkinson, F., & Health for Kids Guideline Development Groups. (2008). Guideline development process for the health for kids in the south east project. Australian Family Physician, 37(6 Spec No), 2-5.
  27. Hasenfeld, R., & Shekelle, P. G. (2003). Is the methodological quality of guidelines declining in the US? comparison of the quality of US agency for health care policy and research (AHCPR) guidelines with those published subsequently. Quality & Safety in Health Care, 12(6), 428-434.
  28. Horlocker, T. T., Neal, J. M., & Rathmell, J. P. (2011). Practice advisories by the american society of regional anesthesia and pain medicine: Grading theevidence and making the grade. Regional Anesthesia and Pain Medicine, 36(1), 1-3. doi:10.1097/AAP.0b013e318203083b
  29. Hsu, J., Brozek, J. L., Terracciano, L., Kreis, J., Compalati, E., Stein, A. T., . . . Schunemann, H. J. (2011). Application of GRADE: Making evidence-based recommendations about diagnostic tests in clinical practice guidelines. Implementation Science : IS, 6(1), 62. doi:10.1186/1748-5908-6-62
  30. Jaouen, R. (2010). The ugly side of practice guidelines. Plastic and Reconstructive Surgery, 125(6), 1848-9; author reply 1849-50. doi:10.1097/PRS.0b013e3181d45bc0
  31. Kaiser, K., Martini, P., Miksch, S., & Ozturk, A. (2007). A meta schema forevidence information in clinical practice guidelines as a basis for decision-making. Studies in Health Technology and Informatics, 129(Pt 2), 925-929.
  32. Kavanagh, B. P. (2009). The GRADE system for rating clinical guidelines. PLoS Medicine, 6(9), e1000094. doi:10.1371/journal.pmed.1000094
  33. Keeley, P. (2004). Making the GRADE. Palliative Medicine, 18(8), 675-676.
  34. Ketola, E., Kaila, M., & Honkanen, M. (2007). Guidelines in context ofevidence. Quality & Safety in Health Care, 16(4), 308-312. doi:10.1136/qshc.2006.019752
  35. Khan, A. R., Khan, S., Zimmerman, V., Baddour, L. M., & Tleyjeh, I. M. (2010). Quality and strength of evidence of the infectious diseases society of america clinical practice guidelines. Clinical Infectious Diseases : An Official Publication of the Infectious Diseases Society of America, 51(10), 1147-1156. doi:10.1086/656735
  36. Kingston, M. E., Krumberger, J. M., & Peruzzi, W. T. (2000). Enhancing outcomes: Guidelines, standards, and protocols. AACN Clinical Issues, 11(3), 363-374.
  37. Knottnerus, J. A., & Tugwell, P. (2011). Standards, guidelines, and norms.<i style="font-family: arial, helvetica, sans-serif; font-size: small; line-heig

Grading: Tables and Scales


Calculators and Tutorials

Interactive Nomogram from the Centre for Evidence Based Medicine

Diagnostic Test Calculator from Alan Schwartz
This calculator can determine diagnostic test characteristics (sensitivity, specificity, likelihood ratios) and/or determine the post-test probability of disease given given the pre-test probability and test characteristics. Given sample sizes, confidence intervals are also computed.

Risk Reduction Calculator from Alan Schwartz
Given information about the probability of an outcome under control and experimental treatments, this calculator produces measures of risk increase/decrease and number needed to treat or harm, including confidence intervals. If some patients were lost to follow-up, the calculator provides estimates for several different scenarios.

Diagnostic Test Cutoffs from Alan Schwartz
A graphical demonstration of the effect of changing cutoff scores on sensitivity and specificity of a test.

Statistical Testing Thresholds from Alan Schwartz
A graphical demonstration like the above, but written in terms of statistical test theory (type I and II error).

PS: Power Sample and Size Calculation from Dale Plummer and William Dupont and Vanderbilt

Evidence Based Practice Tutorial for Nurses from Penn State

Introduction to Evidence-Based Medicine from Duke University Medical Center Library and Health Sciences Library, UNC-Chapel Hill

Introduction to Evidence-Based Medicine from Boston University

Searching the Literature for Evidence-Based Medicine from UCSF

EMBASE Subject Specific Searches