Abstract

Systematic review and meta-analyses (SRMA) form the cornerstone of evidence-based medicine and evidence-based public health. SRMA occupies the highest position in the evidence pyramid. Thus, researchers, clinicians, public health professionals, policymakers, and other stakeholders place much trust in the findings of an SRMA. So, these interpretations must be methodologically robust and statistically sound. No comprehensive and sound tutorial is available for meta-analysis of different effect sizes condensed in a single article. This is an easy yet comprehensive guide to fill this gap. It also incorporates some advanced but essential meta-analytical techniques. Everything has been explained simply in open-source and free software, i.e., R. This allows greater access by researchers from lower- and middle-income countries, ensuring inclusivity and equitable access to health research. Although the primary target readers are beginners of R, seasoned R users can also benefit from the advanced analyses introduced in the meta-analysis, thereby addressing the needs of both groups. It explains statistical concepts briefly as necessary and demonstrates their implementation. The code file for the demonstrated examples and the relevant data extraction sheets have been attached. Then, we have provided examples for reporting the results of a systematic review and meta-analysis, troubleshooting common mistakes, customising the output, changing the finer settings, and exploring other important aspects of meta-analysis. We have finally discussed other sources for further reading for those interested in delving into more advanced and complicated techniques.

Keywords:

Systematic review, Tutorial, Course, Step-by-step guide, Statistical analysis, R Studio, R Software, Troubleshooting, Funnel plot, Egger’s regression, Egger’s test, Doi plot, LFK index, Prediction interval, Heterogeneity, Bubble plot, Meta-regression

References

Sauerland S, Seiler CM. Role of systematic reviews and meta-analysis in evidence-based medicine. World J Surg 2005;29:582–7. https://doi.org/10.1007/s00268-005-7917-7.

Gopalakrishnan S, Ganeshkumar P. Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare. J Family Med Prim Care 2013;2:9–14. https://doi.org/10.4103/2249-4863.109934.

Ferreira González I, Urrútia G, Alonso-Coello P. Systematic Reviews and Meta-Analysis: Scientific Rationale and Interpretation. Revista Española de Cardiología (English Edition) 2011;64:688–96. https://doi.org/10.1016/j.rec.2011.03.027.

Jenicek M. Epidemiology, evidenced-based medicine, and evidence-based public health. J Epidemiol 1997;7:187–97. https://doi.org/10.2188/jea.7.187.

Wang S, Moss JR, Hiller JE. Applicability and transferability of interventions in evidence-based public health. Health Promot Int 2006;21:76–83. https://doi.org/10.1093/heapro/dai025.

Fielding JE, Briss PA. Promoting evidence-based public health policy: can we have better evidence and more action? Health Aff (Millwood) 2006;25:969–78. https://doi.org/10.1377/hlthaff.25.4.969.

Howes F, Doyle J, Jackson N, Waters E. Evidence-based public health: the importance of finding “difficult to locate” public health and health promotion intervention studies for systematic reviews. J Public Health (Bangkok) 2004;26:101–4.

Anderson LM, Brownson RC, Fullilove MT, Teutsch SM, Novick LF, Fielding J, et al. Evidence-based public health policy and practice: promises and limits. Am J Prev Med 2005;28:226–30. https://doi.org/10.1016/j.amepre.2005.02.014.

Cornell University Library. A Guide to Evidence Synthesis: What is Evidence Synthesis? 2023. https://guides.library.cornell.edu/evidence-synthesis/intro (accessed June 18, 2023).

Morton SC, Murad MH, O’Connor E, Lee CS, Booth M, Vandermeer BW, et al. Quantitative Synthesis—An Update. 2008.

García-Perdomo HA. Evidence synthesis and meta-analysis: a practical approach. International Journal of Urological Nursing 2016;10:30–6. https://doi.org/10.1111/ijun.12087.

Ariel de Lima D, Helito CP, de Lima LL, Clazzer R, Gonçalves RK, de Camargo OP. HOW TO PERFORM A META-ANALYSIS: A PRACTICAL STEP-BY-STEP GUIDE USING R SOFTWARE AND RSTUDIO. Acta Ortop Bras 2022;30:e248775. https://doi.org/10.1590/1413-785220223003e248775.

Shim SR, Kim S-J. Intervention meta-analysis: application and practice using R software. Epidemiol Health 2019;41:e2019008. https://doi.org/10.4178/epih.e2019008.

Chen D-G. Meta-analysis for psychiatric research using free software R. Shanghai Arch Psychiatry 2015;27:195–9. https://doi.org/10.11919/j.issn.1002-0829.215063.

Li X, Dusseldorp E, Su X, Meulman JJ. Multiple moderator meta-analysis using the R-package Meta-CART. Behav Res Methods 2020;52:2657–73. https://doi.org/10.3758/s13428-020-01360-0.

Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 2019;22:153–60. https://doi.org/10.1136/ebmental-2019-300117.

Furuya-Kanamori L, Barendregt JJ, Doi SAR. A new improved graphical and quantitative method for detecting bias in meta-analysis. Int J Evid Based Healthc 2018;16:195–203. https://doi.org/10.1097/XEB.0000000000000141.

IntHout J, Ioannidis JPA, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open 2016;6:e010247. https://doi.org/10.1136/bmjopen-2015-010247.

Comprehensive R Archive Network. CRAN Mirrors 2023. https://cran.r-project.org/mirrors.html (accessed June 19, 2023).

Posit. DOWNLOAD RStudio Desktop 2023. https://posit.co/download/rstudio-desktop/ (accessed June 19, 2023).

TC Technology Knowledge Base. Install R and R-Studio for Windows n.d. https://teacherscollege.screenstepslive.com/a/1108074-install-r-and-r-studio-for-windows (accessed June 21, 2023).

TC Technology Knowledge Base. Install R and R-Studio for Mac n.d. https://teacherscollege.screenstepslive.com/a/1135059-install-r-and-r-studio-for-mac (accessed June 21, 2023).

Hands-On Programming with R : Installing R and RStudio n.d. https://rstudio-education.github.io/hopr/starting.html (accessed June 21, 2023).

DataFlair. How to Install R, RStudio and R Packages – The Easiest Process Ever! n.d. https://data-flair.training/blogs/how-to-install-r/ (accessed June 21, 2023).

Viechtbauer W. Conducting Meta-Analyses in R with the metafor Package. J Stat Softw 2010;36. https://doi.org/10.18637/jss.v036.i03.

Schwarzer G, Carpenter J, G R. metasens: Statistical Methods for Sensitivity Analysis in Meta-Analysis. 2023.

Wickham H, Bryan J. readxl: Read Excel Files. 2023. https://doi.org/https://CRAN.R-project.org/package=readxl.

Sedgwick P. How to read a forest plot in a meta-analysis: BMJ 2015:h4028. https://doi.org/10.1136/bmj.h4028.

Dettori JR, Norvell DC, Chapman JR. Seeing the Forest by Looking at the Trees: How to Interpret a Meta-Analysis Forest Plot. Global Spine J 2021;11:614–6. https://doi.org/10.1177/21925682211003889.

Andrade C. Understanding the Basics of Meta-Analysis and How to Read a Forest Plot: As Simple as It Gets. J Clin Psychiatry 2020;81. https://doi.org/10.4088/JCP.20f13698.

Higgins JPT. Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol 2008;37:1158–60. https://doi.org/10.1093/ije/dyn204.

Cochran WG. The Combination of Estimates from Different Experiments. Biometrics 1954;10:101. https://doi.org/10.2307/3001666.

Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539–58. https://doi.org/10.1002/sim.1186.

Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods 2006;11:193–206. https://doi.org/10.1037/1082-989X.11.2.193.

Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med 1997;16:753–68. https://doi.org/10.1002/(sici)1097-0258(19970415)16:7<753::aid-sim494>3.0.co;2-g.

Hedges L V., Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods 1998;3:486–504. https://doi.org/10.1037/1082-989X.3.4.486.

Poole C, Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol 1999;150:469–75. https://doi.org/10.1093/oxfordjournals.aje.a010035.

Borenstein M, Hedges L V, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010;1:97–111. https://doi.org/10.1002/jrsm.12.

Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I(2) in assessing heterogeneity may mislead. BMC Med Res Methodol 2008;8:79. https://doi.org/10.1186/1471-2288-8-79.

Borenstein M, Higgins JPT, Hedges L V, Rothstein HR. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res Synth Methods 2017;8:5–18. https://doi.org/10.1002/jrsm.1230.

Shamim MA. Real-life implications of prevalence meta-analyses? Doi plots and prediction intervals are the answer. Lancet Microbe 2023. https://doi.org/10.1016/S2666-5247(23)00096-4.

Anil A, Shamim MA, Saravanan A, Sandeep M. HPV DNA and p16INK4a positivity in vulvar cancer and vulvar intraepithelial neoplasia. Lancet Oncol 2023;24:e235. https://doi.org/10.1016/S1470-2045(23)00210-3.

Shamim MA, Dwivedi P, Padhi BK. Beyond the Funnel Plot: The Advantages of Doi Plots and Prediction Intervals in Meta-Analyses. Asian J Psychiatr 2023:103550. https://doi.org/10.1016/j.ajp.2023.103550.

Gandhi AP, Satapathy P, Rustagi S, Hermis AH, Sah R, Padhi BK. Comments on “Shigellosis in Southeast Asia: A systematic review and meta-analysis”. Travel Med Infect Dis 2023:102593. https://doi.org/10.1016/j.tmaid.2023.102593.

Thompson SG, Smith TC, Sharp SJ. Investigating underlying risk as a source of heterogeneity in meta-analysis. Stat Med 1997;16:2741–58. https://doi.org/10.1002/(sici)1097-0258(19971215)16:23<2741::aid-sim703>3.0.co;2-0.

Thompson SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ 1994;309:1351–5. https://doi.org/10.1136/bmj.309.6965.1351.

Song F. Exploring heterogeneity in meta-analysis: is the L’Abbé plot useful? J Clin Epidemiol 1999;52:725–30. https://doi.org/10.1016/s0895-4356(99)00066-9.

Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383:257–66. https://doi.org/10.1016/S0140-6736(13)62296-5.

Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2018;11:MR000005. https://doi.org/10.1002/14651858.MR000005.pub4.

Schmucker C, Schell LK, Portalupi S, Oeller P, Cabrera L, Bassler D, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One 2014;9:e114023. https://doi.org/10.1371/journal.pone.0114023.

Page MJ, Sterne JAC, Higgins JPT, Egger M. Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: A review. Res Synth Methods 2021;12:248–59. https://doi.org/10.1002/jrsm.1468.

Viechtbauer W. Publication bias in meta-analysis: Prevention, assessment and adjustments. Psychometrika 2007;72:269–71. https://doi.org/10.1007/s11336-006-1450-y.

Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000;53:207–16. https://doi.org/10.1016/s0895-4356(99)00161-4.

Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta-analysis. Stat Med 2001;20:641–54. https://doi.org/10.1002/sim.698.

Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 2000;53:1119–29. https://doi.org/10.1016/s0895-4356(00)00242-0.

McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 2000;356:1228–31. https://doi.org/10.1016/S0140-6736(00)02786-0.

Rust RT, Lehmann DR, Farley JU. Estimating Publication Bias in Meta-Analysis. Journal of Marketing Research 1990;27:220–6. https://doi.org/10.1177/002224379002700209.

Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics 2000;1:247–62. https://doi.org/10.1093/biostatistics/1.3.247.

Sterne JAC, Egger M. Regression Methods to Detect Publication and Other Bias in Meta-Analysis. Publication Bias in Meta‐Analysis, 2005, p. 99–110. https://doi.org/https://doi.org/10.1002/0470870168.ch6.

Sterne JAC, Harbord RM. Funnel Plots in Meta-analysis. Stata J 2004;4:127–41. https://doi.org/10.1177/1536867X0400400204.

Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629–34. https://doi.org/10.1136/bmj.315.7109.629.

Doi SA. Rendering the Doi plot properly in meta-analysis. Int J Evid Based Healthc 2018;16:242–3. https://doi.org/10.1097/XEB.0000000000000158.

Riley RD, Sutton AJ, Abrams KR, Lambert PC. Sensitivity analyses allowed more appropriate and reliable meta-analysis conclusions for multiple outcomes when missing data was present. J Clin Epidemiol 2004;57:911–24. https://doi.org/10.1016/j.jclinepi.2004.01.018.

Elvik R. Can We Trust the Results of Meta-Analyses?: A Systematic Approach to Sensitivity Analysis in Meta-Analyses. Transp Res Rec 2005;1908:221–9. https://doi.org/10.1177/0361198105190800127.

Mathur MB, VanderWeele TJ. Robust Metrics and Sensitivity Analyses for Meta-analyses of Heterogeneous Effects. Epidemiology 2020;31:356–8. https://doi.org/10.1097/EDE.0000000000001180.

Rücker G, Schwarzer G. Beyond the forest plot: The drapery plot. Res Synth Methods 2021;12:13–9. https://doi.org/10.1002/jrsm.1410.

Stack overflow n.d. https://stackoverflow.com (accessed June 5, 2023).

Stack Exchange n.d. https://stackexchange.com (accessed June 5, 2023).

The R Project for Statistical Computing n.d. https://search.r-project.org (accessed June 5, 2023).

R Seek n.d. https://rseek.org (accessed June 5, 2023).

How to Cite

Shamim, M. A., Gandhi, A. P., Dwivedi, P., & Padhi, B. K. (2023). How to perform meta-analysis in R: a simple yet comprehensive guide. The Evidence, 1(1), 93–113. https://doi.org/10.61505/evidence.2023.1.1.6

License

Copyright© by the author(s). Published by the Evidence Journals. This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Most read articles by the same author(s)