guide to conduct systematic reviews and Meta-Analysis

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
A step by step guide to conduct systematic reviews and Meta-Analysis
(An ICMR Publication)
Anju Sinha, Geetha R. Menon, Denny John

Foreword
I am happy to write this foreword for Beginner’s Guide for Systematic Reviews. This book is mainly aimed at public health and social science researchers for undertaking systematic reviews. The aim of this guide is to promote high standards in commissioning, conducting, and providing practical guidance for undertaking systematic reviews evaluating the effects of health interventions.
Over the last decade, the demand for use of the best available research evidence to inform health care decision making and public policy has increased considerably. Systematic reviews aim to identify, appraise, and summarize the findings from all relevant individual studies on a topic. Well conducted Systematic Reviews provide the best evidence to guide clinical practice, they are considered a cornerstone for the recommendations of Evidence-based practice guidelines and should be an integral part of planning future research activities. The Beginner’s Guide has been written for those with an understanding of health research & seeking skills for conducting systematic reviews. The guide is also aimed at those who commission systematic reviews, such as the government and funding organisations. Though the book is mainly aimed at health practitioners, some researchers working on systematic reviews on social sciences might also find it useful.
I appreciate the efforts of the contributors in preparing the Beginner’s Guide. As we release this guide, I envision that it would serve a variety of audiences such as policymakers, researchers, clinicians and other health professionals who need to be aware of evidence-informed decision-making in health care.
(Dr. Balram Bhargava)

Acknowledgements
This book has been produced through collaborative efforts of the contributing authors. The book draws on the collective learning from the systematic reviewing conducted by the authors, and also from the learning’s that has arisen from interacting with systematic review workshop participants from India (ICMR, DHR, NICPR, NIREH, PGIMER) Nepal, Bangladesh, Ghana, Germany, and Scotland, and researchers embarking on a systematic review from developing countries.
We acknowledge Ms Supreet Kaur, who was a data programmer in the HIV project at ICMR-NIMS when this book was being written, for her inputs in chapter 5 and in preparing a list of textbooks as additional resources in this book.
In addition, the authors are grateful to the reviewers Prof Meenu Singh from PGIMER, Chandigarh and Prof. Sreekumaran Nair, JIPMER, Puducherry who have a rich experience in teaching, training and conducting systematic reviews. Their valuable inputs and review have helped in improving the contents of this book.
We hope this book becomes a useful guide for beginners and early career researchers who are planning to undertake systematic reviews.
Dr. Anju Sinha Dr. Geetha R. Menon Dr. Denny Hohn

Authors
Anju Sinha: Consultant Scientist Division of RBMCH,
ICMR Hqrs, New Delhi. Dr. Anju is a clinician by
training specializing in Epidemiology & Public
Health, with overall 27 years of research experience in
implementation of large community-based field trials
funded by international agencies. At the Indian Council
of Medical Research she worked as a Program Officer in
the area of Neonatal and Child Health, HIV prevention,
and Evidence Based Child Health. She has initiated a
funding scheme on secondary data, supporting systematic reviews and building capacity of Indian scientists in systematic reviews. She is the recipient of the Aubrey Sheiham award in Primary Health Care and the Kenneth Warren award from the Cochrane. She is a Steering Committee member Public Health Evidence South Asia (PHESA), Scientific Advisory Group member International Life Sciences Institute (ILSI), India, member Technical Advisory Committee of Health Technology Assessment India (HTAIn), member Cochrane Neonatal and Cochrane Acute Respiratory Infections Groups, Cochrane Child Health and Public Health Fields & Rapid Review Methods Group and the Campbell Collaboration. She is Director of the Cochrane Affiliate Centre at ICMR Hqrs & Co Chairs the Cochrane India Network. She is involved with Evidence to policy translation, policy briefing, Health Technology Assessments and identifying new research priorities.

Authors
Geetha R. Menon is a Senior Scientist trained as
a Biostatistician working for the ICMR-National
Institute of Medical Statistics. She has an experience
of more than 3 decades and has contributed to
biomedical research both as a primary researcher and
as a research manager. She has a doctorate degree in
Biostatistics and has significant experience of analysing
data from large scale surveys and epidemiological
studies, doing statistical modelling, and undertaking
Systematic Reviews and Meta-Analysis. She has been involved in planning and designing research studies, teaching biostatistics to researchers and medical faculties and conducting capacity building workshops in research methodology, systematic reviews and health economics. Geetha has co- authored over 100 scientific publications in national and international journals and has co-edited two books viz. Road Traffic & Safety (Transportation Issues, Policies and R&D) and Understanding and Treating Head Injuries. She is the recipient of the ISCB Young Scientist Award 2006, Dr. Kelly P O’Keefe Academician & Researcher of the Year Award 2013, Statistical Alliance for
Vital Events (SAVE) – Queen Elizabeth Advanced Scholars (QES) 2018, and Indian Society of Medical Statistics BG Prasad Award 2020.

Authors
Denny John is Adjunct Professor, Faculty of Life and
Allied Health Sciences, Ramaiah University of Applied
Sciences, Bengaluru; and Adjunct Faculty, Amrita
Institute of Medical Sciences & Research Centre, Kochi.
Denny has experience of working across several review
types, such as effectiveness, cost-effectiveness, barriers/
facilitators, prevalence/incidence, risk/aetiology and
diagnostic test accuracy. Denny’s research focusses
mainly on economic evidence in systematic reviews
including equity components. With a background in epidemiology and health economics, he has considerable methodological and statistical expertise, including conducting cost-effectiveness alongside randomized controlled trials and observational studies, as well systematic reviews and meta-analyses. Denny is Chair, Cochrane & Campbell Economic Methods Group (CCEMG), and Advisory Member, Disability Coordinating Group, Campbell Collaboration, and Co-Chair, Early Career Network, Health Technology Assessment International (HTAi). He was on the Editorial Board for ‘Evidence, policy, impact: WHO guide for evidence-informed decision- making’, published by World Health Organization in 2021.

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
List of boxes
Box 1: Quantitative review of domestic violence an example………………….19 Box 2: List of keywords and synonyms related to a question ………………….26 Box 3: Keywords and synonyms for the systematic review example ………27 Box 4: Sources of published literature……………………………………………………..28 Box 5: Sources of grey literature ……………………………………………………………..30 Box 6: Good practice for data extraction………………………………………………….32 Box 7: Description of critical appraisal process ……………………………………….52 Box 8: Plain language summary example………………………………………………..81
List of figures
Figure 1:
Figure 2: Figure 3:
Figure 4: Figure 5: Figure 6: Figure 7: Figure 8: Figure 9:
Conceptual model of stakeholders, identified by the actors, their roles and their actions …………………………………………………….6 Common stages of a systematic review …………………………………15 Structured questions for systematic reviews and relation between question components in a comparative study …………19 LÁbbe Plot…………………………………………………………………………….64 Galbraith Plot………………………………………………………………………..65 Baujat Plot……………………………………………………………………………..66 Funnel Plot ……………………………………………………………………………67 Forest plot comparing two interventions ……………………………..68 PRISMA 2020 Flow diagram …………………………………………………71
List of tables
Table 1 : Table 2: Table 3:
Table 4: Table 5: Table 6: Table 7: Table 8: Table 9: Table 10:
Differences between a narrative review and a systematic review……………………………………………………………………………………..2
Characteristics of narrative reviews, scoping
reviews and systematic reviews………………………………………………4
Software for various processes in the systematic review……………………………………………………………………………………..7
Types of review questions …………………………………………………….16 The PICO process ………………………………………………………………….17 Types of reviews……………………………………………………………………23 Saving a search strategy and the number of records ……………..31 Critical appraisal of cohort studies example………………………….52 GRADE certainty ratings ………………………………………………………55 Summary of findings …………………………………………………………….56

Contents
Foreword ………………………………………………………………………………………………… iii Acknowledgements …………………………………………………………………………………….. v
Authors ……………………………………………………………………………………………….. vii List of boxes……………………………………………………………………………………………….. x List of figures …………………………………………………………………………………………….. x List of tables ………………………………………………………………………………………………. x

Introduction……………………………………………………………………………………… 1
1.1 What Is a Systematic Review?…………………………………………………… 1
1.2 Why do we need a systematic review?……………………………………… 1
1.3 What is the difference between a narrative
review and a systematic review?………………………………………………. 2
1.4 Where to find systematic reviews?……………………………………………. 4
Getting Started …………………………………………………………………………………. 6 2.1 Writing a Systematic Review Protocol ……………………………………… 9 2.2 Components of a Systematic Review protocol …………………………. 9
Steps in a systematic review …………………………………………………………… 15
3.1 Structure an answerable and focussed review question…………..16
3.2 Types of systematic reviews……………………………………………………. 20
3.3 Identify relevant studies through a comprehensive
search strategy………………………………………………………………………… 25
3.4 Undertake a comprehensive search ………………………………………… 28
3.5 Select databases……………………………………………………………………….28
3.6 Tailor search strategy to database(s) ……………………………………….. 30
3.7 Save search and export results………………………………………………… 31
3.8 Select studies for inclusion based on pre-defined criteria ……….. 31
3.9 Description of study characteristics ………………………………………… 32
3.10 Quality assessment using critical appraisal tools ……………………. 51
3.11 Quality of evidence…………………………………………………………………. 54
Meta-Analysis ………………………………………………………………………………… 59
4.1 Effect measures ………………………………………………………………………. 60
4.2 Heterogeneity …………………………………………………………………………. 62

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
4.3 Graphical methods for identification of heterogeneity ……………. 64
4.4 Meta-regression………………………………………………………………………. 67
4.5 Presentation ……………………………………………………………………………. 68
4.6 Reporting a systematic review and Meta-Analysis …………………. 70

Systematic review and Meta-Analysis software ……………………………… 72 5.1 Systematic review software…………………………………………………….72 5.2 Meta-Analysis software ………………………………………………………….. 75
Disseminating Systematic Review Findings …………………………………… 79
6.1 Evidence Summary…………………………………………………………………. 79
6.2 Plain Language Summary (PLS)……………………………………………… 80
6.3 Policy briefs ……………………………………………………………………………. 82
References ……………………………………………………………………………………………… 84 Other Resources …………………………………………………………………………………….. 87
Introduction
1.1 What Is a Systematic Review?
A systematic review is a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the reviews1.
Diverse range of information sources and an explosion of knowledge have made it impossible for clinical researchers to stay abreast with advances in a given field. Systematic reviews and meta-analyses were conducted in health in 1970s and 1980s. Both the terms meta-analyses and systematic reviews were used interchangeably and often created confusion among the readers. Chalmers and Altman2 suggested that the term ‘Meta-Analysis’ may be used only for statistical synthesis or quantitative methods of combining the evidence from individual studies and it may or may not be part of a systematic review.
Systematic reviews are most often used to study the effectiveness of a particular drug treatment compared with a placebo or any other alternative treatment. However, these cover a wide range of other issues like:
• Surgical and nursing techniques e.g. the best ways of carrying out knee replacements or the best methods of dressing for chronic wounds
• Psychosocial interventions e.g. community-based interventions for people with schizophrenia
• Public-health interventions e.g. impact of mobile health (mHealth) interventions in health care delivery or lifestyle interventions in reducing the prevalence of type II diabetes
• Adverse effects of drugs or other treatments
• Economic evaluations e.g. evaluation of implementation intervention in public health or drug trials to identify which intervention is cost effective
• Although Systematic Reviews are predominantly used in intervention studies, researchers also use this technique in economic evaluation, diagnostics tests accuracy, prevalence/incidence etc.
1.2 Why do we need a systematic review?
Systematic reviews are needed for the following reasons
a. To keep abreast of all previous and new research

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
b. To introduce a new treatment that is expected to be better than an existing one
c. To discontinue an old treatment which might be out dated, harmful or less cost effective
d. To draft guidelines for health and social interventions or treatment management
e. To arrive at a consensus where conflicting evidence is reported
1.3 What is the difference between a narrative review and a systematic review?
In comparison with literature or traditional narrative reviews, systematic reviews are much time-intensive and need a research team with multiple skills and contributions. Table 1 provides a description of these differences.
There are some cases where systematic reviews are unable to meet the necessary objectives of the review question. In such a case, scoping reviews (which are sometimes called scoping exercises/scoping studies) may be more useful to consider. Table 2 provides the characteristics of narrative reviews, scoping reviews and systematic reviews.
Table 1 : Differences between a narrative review and a systematic review
Narrative Review
Systematic Review
Goals
Provides summary or overview of topic
Answers a focussed review question
Question
Can have a broad topic or a specific question. Hypothesis might not be stated.
Clearly defined review question using PICO as a guide. Hypothesis is stated.
Authors
One or more
Three or more
Protocol
No protocol
A peer review protocol or plan is included
Objectives
May or may not be identified
Has clearly stated objectives
Inclusion/ exclusion criteria
Criteria not usually specified
Criteria stated before the review is conducted
Search strategy
No detailed search strategy, mostly conducted using keywords and snow-balling
Detailed and comprehensive search strategy
2

Introduction
Narrative Review
Systematic Review
Sources of literature
Non-exhaustive and not stated always. Prone to publication bias.
List of databases, grey literature and other sources are considered.
Selection criteria
Usually subjective or no selection criteria. Prone to selection bias.
Selection process usually clear and explicit
Appraisal of study quality
Variation in evaluation of study quality of studies
Use of standard checklists for rigorous appraisal of study quality
Extracting relevant information
Not explicit and clear
Clear and specific
Synthesis
Summary based on studies which have not been checked for quality and can be influenced by the reviewers needs and beliefs
Clear summaries of studies based on high quality evidence Narrative, quantitative or qualitative synthesis
Conclusions
Sometimes evidence based but could be prone to researcher bias (influence of author’s personal belief)
Evidence-based
Timeline
Weeks to months
Months to years
Requirements
Understanding of topic, and searching of 2-3 databases
At least one of the authors with good knowledge of the topic, searches done for all relevant databases
Value
Provides summary of literature on the topic
Conclusions may be subjective hence minimal reproducibility of findings
Cannot be continuously updated
Provides high-quality evidence, and supports evidence-based practice
Detailed and accurate documentation of methods using PRISMA means results can be reproduced Periodically updated to include new evidence
3

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
Table 2: Characteristics of narrative reviews, scoping reviews and systematic reviews
Narrative Reviews
Scoping Reviews
Systematic Reviews
A priori review protocol
No
Yes (some)
Yes
PROSPERO registration of the review protocol
No
Noa
Yes
Explicit, transparent, peer reviewed search strategy
No
Yes
Yes
Standardized data extraction forms
No
Yes
Yes
Mandatory Critical Appraisal (Risk of Bias Assessment)
No
Nob
Yes
Synthesis of findings from individual studies and the generation of ‘summary findings’
No
No
Yesc
a Current situation; this may change in time.
b Critical appraisal is not mandatory; however, reviewers may decide to assess and report the risk of bias in scoping reviews.
c by using statistical Meta-Analysis (for quantitative effectiveness, or prevalence or incidence, diagnostic accuracy, aetiology or risk, prognostic or psychometric data), or meta-synthesis (experiential or expert opinion data) or both in mixed method reviews3.
1.4 Where to find systematic reviews?
Before undertaking a systematic review, it is necessary to look for ongoing reviews or completed reviews on the topic of interest. Below are some useful websites to start searching for systematic reviews.
• Database of Abstracts of Reviews of Effects (DARE) https://www.crd.york.ac.uk/ CRDWeb/HomePage.asp
• Cochrane Database of Systematic reviews https://www.cochranelibrary.com/cdsr/reviews
• National Institute for Health and Care Excellence UK Database of Uncertainties about the Effects of Treatments (DUETs) ; https://www.evidence.nhs.uk/
4

Introduction
• National Institute for Health Research – Health technology Assessment NIHR- HTA
https://www.nihr.ac.uk/explore-nihr/funding-programmes/health-technology- assessment.htm
• Campbell Library of Systematic Reviews https://campbellcollaboration.org/better-evidence.html
• Evidence for policy and Practice Information (EPPI ) Centre – https://eppi.ioe. ac.uk/webdatabases4/Intro.aspx?ID=9
• Database of promoting health effectiveness reviews : https://eppi.ioe.ac.uk/ webdatabases4/Intro.aspx?ID=9
• Agency for Healthcare Research and Quality https://www.ahrq.gov/research/findings/evidence-based-reports/search.html
• Scottish Intercollegiate Guidelines Network: https://www.sign.ac.uk/our- guidelines.html
• BMJ Evidence based medicine:https://ebm.bmj.com/pages/collections/ebm_ verdict/
• PROSPERO International prospective register of systematic reviews
• Turning Research into practice (TRIP): https://www.tripdatabase.com/
• Search filters in major databases e.g. Medline, EMBASE, PSYCHLIT, CINAHL (for search filters see ‘searching for specific study types’ below)
Summary
• A systematic review is a study that identifies a specific review question, identifies relevant studies using a comprehensive search strategy, appraises the quality of these studies and summarizes their results in a scientific manner
• Systematic reviews can be conducted on many different types of primary studies
• Systematic reviews can be used for informing policies that impact quality, safety and values of health care
• Systematic reviews provide summary evidence from available literature but narrative reviews do not follow scientific review methodology
5

Getting Started
Before initiating a systematic review, it is important to consider four main aspects in managing the review:
a) Formation of a review team- A systematic review team should include experts with a range of skills including expertise in information retrieval, epidemiologist or clinical expert, systematic review methods, statistics, and other aspects e.g. health economist if required for cost- effectiveness/cost-benefit analysis reviews and qualitative expects for research methods where appropriate.
b) Formation of an advisory group – An advisory group including health care professionals, patient representatives, service users and experts in research methods who may be consulted at key stages may be necessary for the funding agencies.
c) Timeline- The timelines for completing various evidence synthesis activities may vary. However, organizations such as Cochrane and Campbell Collaboration suggest completing a review within a year.
d) Stakeholder engagement- Various studies have emphasized the importance of engaging stakeholders to ensure that systematic reviews are shaped by members from the policy and practice community who would be using them4,5. Figure 1 provides a conceptual model of stakeholders and the actors.
e) Software- The selection of software will need to be considered for various stages of the systematic review process. Table 3 provides a list of some of the software applications found useful at various stages of the review.
Figure 1: Conceptual model of stakeholders, identified by the actors, their roles and their actions
Actors
Advocacy groups Business
Citizens Decision-enforcers Decision-makers Publishers Research funders Researchers
Roles
Editors/peer-reviewers Endorsers
Evidence holders Funders
Publishers Communicators Question askers Reviewers
Scope influencers
Service providers
Service users
Users of the review
Actions
Suggest sources of literature Submit articles
Undertake the review Endorse the review
Facilitate access to the review Read the review
Share the review
Integrate findings into decisions
Set the review’s methodological standards
Provide funding and/or in-kind contributions
Share knowledge and experience of scope and context

Example
Policy maker
Policy focus component of deworming
Getting Started
Uses the review finding to justify policy decisions on use of deworming in a country
Table 3: Software for various processes in the systematic review
Activity
Software
Cost
Organisation
Reference management
Zotero
Mendeley EndNote Reference Manager EPPI Reviewer
Free Paid Paid Paid Paid
RRCHNM Elsevier
Clarivate Analytics Thomson Reuters EPPI Centre
Screening
Covidence EPPI Reviewer Rayyan MS-Excels SUMARI DistillerSR
EPPI Reviewer
Free/Paid Paid
Free
Free
Free
Free for students, 4 mnths
Paid
Covidence
EPPI Centre Qatar Foundation Microsoft
JBI
Evidence Partners
EPPI Centre
Coding
EPPI Reviewer MS-Excel SUMARI
Paid Free Free
EPPI Centre Microsoft JBI
Data extraction
EPPI Reviewer MS-Excel$ SRDR SUMARI DistillerSR
Paid Free Free Free
Free for students, 4 months
EPPI Centre Microsoft
CEBM
JBI
Evidence Partners
Critical Appraisal
EPPI Reviewer MS-Excel QARI MAStARI ACTUARI
Paid Free Free Free Free
EPPI Centre Microsoft JBI
JBI
JBI
7

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
Activity
Software
Cost
Organisation
Meta-Analysis
RevMan
STATA SPSS
R
CMA MedCalc
Mix 2.0
OpenMeta Analyst MAStARI
Free Paid Paid Free Paid Paid Paid Free Free
Cochrane STATA Corp SPSS Inc CRAN CMA Corp MedCalc Inc Biostat XL CEBM
JBI
Qualitative
nVivo QARI
Paid Free
QSR International JBI
Mixed Methods
MAXQDA
Paid
VERBI GmbH
$MS-Excel would need to be formatted to conduct the various components of the review process. JBI softwares are available only for reviews with the Joanna Briggs Institute (JBI), Australia.
The following sites have some learning resources for beginners:
• Campbell Collaboration: https://www.campbellcollaboration.org/
• Cochrane: https://www.cochrane.org/
• EPPI-Centre: https://eppi.ioe.ac.uk/cms/
• Systematic Review Toolbox: http://systematicreviewtools.com/
• International Initiative for Impact Evaluation (3ie): https://www.3ieimpact.org/
• Centre for Evidence Based Medicine https://www.cebm.net/
• Centre for Reviews and Dissemination Databases: https://www.crd.york.ac.uk/ crdweb/
• Health Evidence Network (HEN), WHO: https://www.euro.who.int/en/data-and- evidence/evidence-informed-policy-making/health-evidence-network-hen
• Africa Evidence Network: www. https://www.africaevidencenetwork.org/en/
• The EQUTOR Network: https://www.equator-network.org/
• The GRADE Working Group: www. https://gradeworkinggroup.org/
• NIHR HTA: https://www.journalslibrary.nihr.ac.uk/#/
8

2.1 Writing a Systematic Review Protocol
A review protocol is a guide for a well written systematic review. It explains the rationale for conducting the systematic review, states the hypothesis and outlines the methodology to be used. It is important to note that the protocol is a priori statement of aims and methods of the systematic review, and referred back to whenever is needed during the systematic review process. Research question(s), aims and methods are considered in advance to identify the relevant literature to ensure the conduct of the review with minimal bias, access to peer review, greater efficiency in review process6. Protocol development is often an iterative process that requires discussion within the review team, advisory group and sometimes with the funding agency. Peer review and publication makes the protocol publicly available.
A Cochrane review protocol is considered as an individual publication. Non- Cochrane protocols should be registered on PROSPERO – an international database of prospectively registered systematic reviews in health and social care7. Key features from the review protocol are recorded and maintained as a permanent record. Systematic reviews should be registered at inception (i.e. at the protocol stage) to help avoid unplanned duplication and to enable the comparison of reported review methods with what was planned in the protocol8. This prevents duplication (research waste) and makes the process easy when the full systematic review is sent for publication.
By writing a protocol and adhering to it during the review process, the researcher makes it clear that the decisions taken while conducting the review are not arbitrary, the decision to include or exclude studies in the review are not guided by individual choices or prejudices (bias) of the authors or prior knowledge about their results. There are several resources available for the beginners such as the Cochrane Handbook9, PRISMA Extension for protocols (PRISMA-P)10, Institute of Medicine-Standards for systematic reviews11 etc.
Systematic Review should be undertaken by a team of individuals with different areas of expertise. The team should consist of a person with clinical expertise, a person with systematic review experience, a methods person with statistical expertise, and someone with multidisciplinary experience. Someone from the relevant location/population allows double-checking of inclusion of studies and data collection.
2.2 Components of a Systematic Review protocol
The protocol for a systematic review is written using the following format.
Getting Started
9

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
Background: This section for a Cochrane systematic review has a structured format. Under the sub-heading Description of the condition, the health condition/disease under consideration is described with definitions and epidemiological information as evidenced from latest research.
Description of the intervention (for intervention reviews) has a description of the intervention to be evaluated in the systematic review; recent publications on this intervention from the literature should be aligned to the review. The mechanism of action of the intervention is described in a paragraph: How the intervention might work, followed by justification for conducting the systematic review under: why it is important to do this review. In this section the authors are required to mention about other published systematic reviews on the topic, if there are any, the gaps therein that would be addressed in the present review.
Objectives: The primary and secondary objectives of the review are stated under objectives. Additionally, the pre-specified sub-group analyses within the major comparisons need to be listed here.
Methods: This section of the protocol elaborates on the criteria for considering studies for the review (inclusion/exclusion criteria): types of studies (randomized, quasi-randomized, cluster-randomized in intervention reviews), types of participants (the study population, age groups, sex, gestation etc. need to be described) types of intervention (the formulation, mode of administration, doses etc. need to be mentioned) and types of outcome measures: Primary, secondary under different comparison should be described, The procedure to be followed for screening of titles abstracts and full text articles requires that all the steps are completed by two persons. In case of disagreements, a third person (usually more experienced) involved as a team member is approached for resolution. Reviewers may choose to perform a blinded review of the articles retrieved through search.
Search Methods for identification of studies: A comprehensive and up to date and reproducible search is a hallmark of a systematic review. This includes electronic Searches as well as searching other resources (hand searching, non-indexed journals, conference proceedings and grey literature, cross-references/citation searching/manufacturers/personal contacts). It is a pre-requisite that at least 2-3 electronic databases are searched, in order to qualify as a systematic review. The reviewers need to decide what databases (MEDLINE via Pub med, EMBASE, Cochrane central register of controlled trials (CENTRAL) Cumulative Index to Nursing and Allied Health Literature (CINAHL) and sources will be searched, in the context of their topic. The time period for search needs to be specified, search terms and key words have to be written and a search strategy needs to be written down. Search is
10

Getting Started
a complex activity in the conduct of a systematic review, requiring skills as well as access to databases, and resources. It is recommended to seek help from an information specialist/librarian. Cochrane review groups assist the review authors with searching through their designated search coordinators. It is advisable to consider each of the components of the Participants, Interventions, Comparator/comparison and Outcomes (PICO) to derive the search terms. It is important to mention how the reviewers would search for unpublished data (grey literature), conference abstracts. The name of the person who will run the searches, if known, should also be mentioned.
Searching regional databases (e.g. IndMed), clinical trial registries (e.g. CTRI), hand-searching of non-indexed journals, conference proceedings and unpublished (grey) literature should be attempted by the reviewers and described in the methodology.
Data Collection and Analysis: This section comprises of the following sub- headings and their description.
Selection of studies: In a systematic review all steps are performed by two reviewers independently. This is crucial for avoiding personal biases at any step of the conduct of the review. Selection of studies is done as per the laid down inclusion exclusion criteria. Two review authors independently review the titles and abstracts of articles identified by searches for eligibility. Studies are classified as included, excluded or unclear. Full articles are retrieved after title and abstract screening to evaluate whether the study should be included or not based on the PICO of the review. Disagreements between the two reviewers are resolved by discussion or consultation with the third reviewer.
Data Extraction and management: Two review authors should independently extract data from the included studies on a predesigned and pretested data extraction sheet. Authors of the original studies may be contacted in case of incomplete information in the published article included in the review. The data to be extracted include general information (study ID, date of extraction, title, authors, and source of study if not published); study characteristics (study design, participants and inclusion/exclusion criteria used in the study); details of interventions (Including doses, treatment duration, comparison details, and duration of follow up).
Assessment of Risk of Bias in included studies: ‘Risk of bias assessment tool and criteria are described in the Cochrane Handbook for Systematic Reviews of Interventions and are used to assess risk of bias for included studies. Two review authors should independently assess risk of bias in the included studies by assessing randomisation sequence generation; allocation
11

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
concealment; blinding of participants, personnel, and outcome assessors; incomplete outcome data; selective outcome reporting; and other sources of bias.
Measures of Treatment effect: Methods of statistical analyses are detailed under this sub-heading. The Cochrane Handbook provides a detailed description of the analytical methods to be used for dichotomous viz. risk ratios (RRs) and risk differences (RDs) with 95% confidence intervals (CIs). For continuous outcomes, measures of effect as weighted mean differences (MDs) with 95% CIs and standardised mean difference (SMD) should be reported.
Unit of analysis issues: Analyses should consider the level at which randomisation was done -individual or cluster. In the event that cluster- randomised studies are going to be included appropriate adjustment for clustering would be required (multiply the standard error derived from the confidence interval of the effect estimate by the square root of the design effect). The generic inverse variance method in Review Manager 5.3 software can perform Meta-Analysis using inflated variances.
Dealing with missing data: Authors of original trials included in the review should be contacted in case of incomplete/missing data
Assessment of heterogeneity: Statistical heterogeneity can be assessed via visual inspection of forest plots of included trials, using the 2 test and the I2 statistic.Trial characteristics(participants,design,interventions,outcomes, and risk of bias) are examined to identify the source of any observed heterogeneity. There are cut-offs recommended by Cochrane review groups for results of the I2 test: < 25% none, 25% to 49% low, 50% to 74% moderate, and 75%+ high heterogeneity12.
Assessment of reporting biases: Reporting biases are assessed by trying to identify whether the study was included in a trial registry, whether a protocol was published, and whether the methods section provides a list of outcomes. The reported outcomes can be compared with what has been mentioned in the trial protocol by the trial authors versus the outcomes reported in the published article.
Data Synthesis: In this section the systematic reviewers describe the statistical methods of combining data from included studies in a Meta- Analysis if possible. However, the studies should be similar, to be combined in the Meta-Analysis. Statistical guidelines are available in the Cochrane Handbook for Systematic Reviews of Interventions9. The Rev-Man software13, free to download can be used for conducting the analyses. A fixed-effect
12

Getting Started
or a random-effects model may be chosen as appropriate. In case of high heterogeneity Meta-Analysis is not recommended, only a narrative summary of trial findings may be provided.
Risk of bias: Each individual study included in a systematic review should be assessed for key sources of bias. Selection bias could occur due to systematic differences in baseline characteristics between the groups compared in a study, or in randomized trials, from an inadequate generation of a random allocation sequence or inadequate concealment of allocations before group assignment. Other biases such as detection bias, performance bias, attrition bias and outcome reporting bias can also occur. It is important that review authors report the methods used to assess the risk of bias in individual studies, as well as the findings of the assessment. The Cochrane Collaboration’s tool for assessing the risk of bias in a systematic review, each study is graded as low (-), high (+) or unclear (?) across different types of bias using a domain-based qualitative description of critical areas of potential bias in clinical trials. For meta-analyses, authors can conduct sensitivity analyses that exclude trials at high risk of bias to determine the effect on the results.
Quality of evidence: GRADE approach, as outlined in the GRADE Handbook, is used to assess the quality of evidence for the clinically relevant outcomes14. Two review authors independently assess the quality of evidence for each outcome. Evidence from RCTs is initially considered high but may be downgraded one level for serious (or two levels for very serious) limitations on the basis of the following: design (risk of bias), consistency across studies, directness of evidence, precision of estimates, and presence of publication bias. A software ‘GRADE pro GDT’ is used to create a ‘Summary of findings’ table to report the quality of the evidence. The GRADE approach yields an assessment of the quality of a body of evidence according to one of four grades. 1. High: We are very confident that the true effect lies close to that of the estimate of the effect. 2. Moderate: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. 3. Low: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect. 4. Very low: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.
Subgroup analysis and investigation of heterogeneity: Sub-group analyses are conducted to explore the reason for heterogeneity detected. Review authors should a priori determine and describe the possible sub-groups in their review under the comparisons envisaged.
13

BEGINNER’S GUIDE FOR SYSTEMATIC REVIEWS
Sensitivity analysis: Sensitivity analyses are conducted to assess the impact of high risk of bias on the outcome of meta-analyses by adding studies with high risk of bias to pooled studies with low risk of bias. Similarly, other assumptions can also be changed to see their changing effect on the overall estimate.
References: A list of references of studies included, excluded in the review and additional references used in the background section should be provided. The study ID usually comprises of the Surname of the author and the year of publication.
Acknowledgement: Authors may list the names of people who have helped them in the process of conducting the systematic review but who do not qualify as authors.
Appendices: The detailed search strategy is usually included in this section
Contribution of Authors: The contribution of each review author should be mentioned in this section.
Declaration of interest: Any conflicts of interests should be stated in this section
Sources of Support: If the authors have received funding to conduct the systematic review, they should mention it here. If not, they should acknowledge the internal support of their respective organizations.
Example of a Systematic Review Protocol: Sinha A, Pradhan A, Thumburu KK, Gupta N. Probiotics for the prevention or treatment of hyperbilirubinemia in late preterm and term neonates. Cochrane Database of Systematic Reviews 2017, Issue 8. Art. No.: CD012781. DOI: 10.1002/14651858.CD012781
14

Steps in a systematic review
A systematic review is based on a clearly formulated question, identifies relevant studies, appraises the quality and summarizes the evidence by an explicit methodology. Figure 2 shows the common stages in a systematic review.
Figure 2: Common stages of a systematic review