Systematic reviews

Introduction to the topic

KNOWLEDGE SYNTHESIS – a research method involving the collection, evaluation, and analysis of existing studies on a specific topic. It leads to the development of a final interpretation.

SYSTEMATIC REVIEW – the most comprehensive and structured form of knowledge synthesis, involving the analysis of available literature addressing a specific, precisely formulated research question.

Charakterystyka:

META-ANALYSIS – a quantitative method for synthesizing the results of multiple scientific studies, often serving as an optional but highly valuable component of systematic reviews. It enables the integration of data from different sources and the estimation of the overall effect of the intervention being analyzed, as well as the assessment of the strength and consistency of the results obtained.

Advantages of a systematic review:

Systematic Review vs. Traditional Review

NORMS TRADITIONAL REVIEW SYSTEMATIC REVIEW
PurposeProvides a general overview of the topic.Has a clearly defined objective and a specific research question (including population, intervention, and outcome).
Review planningNo predefined plan; considerable flexibility for the author.A clearly defined and documented process.
Searching for studiesAn unspecified search strategy, usually not covering all relevant publications.A carefully developed strategy covering multiple databases and grey literature; a comprehensive search for all available studies.

Other Types of Reviews

TYPE OF REVIEWCHARACTERISTICS
Depending on the type of data analyzed
QUANTITATIVE REVIEWFocuses on numerical data and measurable study outcomes; most often includes a meta-analysis.
QUALITATIVE REVIEWDeals with non-numerical data; uses thematic or narrative synthesis methods.
MIXED METHODS REVIEWCombines quantitative and qualitative approaches in a single analysis.
Other examples of knowledge synthesis
RAPID REVIEWAn accelerated form of systematic review in which selected stages are simplified or omitted to obtain an answer quickly.
SCOPING REVIEWMaps the extent of available knowledge on a given topic, including the number, types, and characteristics of studies.
REALIST REVIEWAnalyzes how, why, and under what conditions an intervention works or does not work, taking context, mechanisms, and outcomes into account.
UMBRELLA REVIEWA review of systematic reviews that synthesizes the findings of multiple previous analyses.
LIVING REVIEWA systematic review that is continuously updated by incorporating the latest research findings.

Stages of conducting reviews

1. Defining the Research Question

RESEARCH QUESTION – the starting point of every systematic review and scientific literature analysis. Its purpose is to precisely define what the researcher wants to find out and what data will be needed to answer the question. Its formulation determines the selection of studies, databases, and the way in which the results will be interpreted.

In clinical research, the model PICO is most commonly used. It helps structure the research question in a logical and clear way:

P Population Population or study group
I Intervention Intervention or action
C Comparison Comparison (e.g., with another method or no intervention)
O Outcome Expected outcome or effect

Depending on the type and topic of the research, other models for formulating research questions are also used.

2. Establishing Study Inclusion and Exclusion Criteria

STUDY ELIGIBILITY CRITERIA – a set of clearly defined rules used to determine whether a study should be included in the review.

They should ideally be established at the initial stage of formulating the research question, when the scope of the planned review is being defined. These criteria form the basis for study selection after studies have been identified through the literature search using the previously developed search strategy.

INCLUSION CRITERIA EXCLUSION CRITERIA
DefinitionRules specifying which studies should be included in the review.Rules specifying which studies should be rejected and excluded from the analysis.
PurposeEnable the selection of publications that best address the research question and the objective of the review.Allow the elimination of publications that do not meet the methodological or thematic requirements.
Methodological significanceEnsure consistency and transparency in the study selection process.Reduce the risk of including inappropriate or unreliable sources (systematic bias).
3. Selecting Information Sources and Databases

DATABASE SELECTION affects the completeness and reliability of a review. Therefore, in the field of medical sciences, it is recommended to use established databases such as PubMed/MEDLINE, Embase and the Cochrane Library . It is also important to include grey literature, which may contain valuable unpublished or difficult-to-access research findings.



PRINCIPLES FOR SELECTING INFORMATION SOURCES:

  1. Tailor the databases to the research field.
  2. Consider the type of studies and the language of publications.
  3. Apply the balance principle:
    • at least 2 recommended databases (e.g., PubMed and Embase) + 1 specialized database,
    • at least 2 grey literature sources.
  4. Check the availability of full-text articles (some databases have embargoes or licensing restrictions).
4. Developing the Search Strategy

SEARCH STRATEGY – a set of keywords, synonyms, and Boolean operators used to identify all relevant studies.

Good practices include:

  • documenting the search process (strategies, dates, number of results retrieved from each database),
  • creating accounts in databases (to facilitate saving searches, accessing search history, and setting alerts),
  • setting up alerts to monitor newly published studies,
  • applying search limits only when justified (e.g., date, language, or study type).

BUILDING A SEARCH STRATEGY IN 7 STEPS:

  1. Define the research question.
  2. Identify its key concepts.
  3. Find corresponding terms in controlled vocabularies MeSH / Emtree).
  4. Add synonyms and alternative forms of the terms.
  5. Apply Boolean operators, truncation, and wildcards.
  6. Adapt the search query to the requirements and specific features of the selected database.
  7. Document the search strategy.
5. Study selection

STUDY SELECTION – a key stage of a systematic review, aimed at selecting only those publications that meet the previously established inclusion and exclusion criteria.

The study selection process should be transparent, reproducible, and consistent with the PRISMA Guidelines.

The pathway from identification to inclusion is illustrated by the PRISMA Flow Diagram.

STAGES OF THE STUDY SELECTION PROCESS:

1 Importing search results from databases After collecting the records identified from different information sources and databases.
2 Removing duplicates Automatically or manually.
3 Title and abstract screening Excluding studies that are not relevant to the research question.
4 Full-text screening Assessing whether the study actually meets the established inclusion criteria.
5 Inclusion in the review Only publications meeting all eligibility requirements are included.
6. Data Extraction from Selected Studies

DATA EXTRACTION – the systematic collection of information from studies included in a systematic review. Data are collected according to a predefined framework, ensuring consistency, comparability, and organization.

EXAMPLE DATA EXTRACTION PROCESS:

1 Preparing a data extraction form Defining what information will be collected (e.g., population, intervention, outcomes, methodology).
2 Identifying relevant data in publications Selecting key information from the full texts of the studies.
3 Systematically extracting data Collecting data according to the predefined framework.
4 Verifying data accuracy Checking the completeness of the data and its consistency with the original publications.
5 Organizing and standardizing data Preparing the data for further analysis and synthesis of results.
7. Assessing Risk of Bias and Study Quality

RISK OF BIAS AND STUDY QUALITY ASSESSMENT – a key stage of a systematic review aimed at determining the credibility and reliability of the included publications. It helps assess the extent to which the results may be affected by bias and whether they can provide a sound basis for the synthesis of scientific evidence.

SYSTEMATIC BIAS (ang. bias)a consistent distortion of study results resulting from its design, conduct, or analysis, which may lead to incorrect conclusions.

IMPORTANCE OF THIS STAGE IN SYSTEMATIC REVIEWS:

  • enables the assessment of the credibility and strength of scientific evidence,
  • identifies and reduces the impact of bias on study results,
  • helps prevent the inclusion of studies with low methodological quality,
  • facilitates better interpretation and comparison of study results,
  • increases the reliability of conclusions drawn from systematic reviews and meta-analyses.

Tools for Detecting Bias and Assessing Study Quality

8. Analysis and Interpretation of Results

ANALYSIS AND INTERPRETATION OF RESULTS – a stage of the systematic review in which the data collected from the studies are organized, compared, and synthesized to answer the research question. Depending on the type of data, this may take the form of qualitativequantitative (meta-analysis) synthesis.

THIS STAGE INCLUDES, AMONG OTHER THINGS:

  • collating and comparing the results of individual studies,
  • identifying similarities and differences between the results,
  • assessing the consistency and heterogeneity of the data,
  • conducting a qualitative (descriptive) or quantitative (meta-analysis) synthesis,
  • interpreting the results in the context of study quality and risk of bias,
  • formulating conclusions that address the research question.
9. Dissemination of Results as a Systematic Review

DISSEMINATION OF RESULTS – the final stage of a systematic review, involving the presentation and sharing of the findings in a clear and standards-compliant format. Its purpose is to communicate reliable conclusions to the scientific community, clinical practitioners, and other interested audiences.


THIS STAGE INCLUDES, AMONG OTHER THINGS:

  • preparing a full report or scientific article,
  • presenting the methodology and results in accordance with PRISMA guidelines,
  • preparing tables, graphs, and diagrams (e.g., PRISMA Flow Diagram, forest plot),
  • publishing the results in a scientific journal or repository,
  • presenting the findings at conferences, training sessions, or seminars,
  • sharing conclusions that support clinical practice and further research.

PRISMA guidelines

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) – a set of guidelines for systematic reviews and meta-analyses that help authors report the methodology and results of their research in a clear, transparent, and comprehensive manner.

IMPORTANCE OF PRISMA GUIDELINES FOR SYSTEMATIC REVIEWS:

Access to the Guidelines: PRISMA 2020


Databases and Registries
DATABASE/REGISTRYDESCRIPTION
PubMed / MEDLINEThe most important biomedical database containing peer-reviewed journals; a standard resource for clinical research and RCTs.
EmbaseExpands on PubMed with conference abstracts and broader coverage of European literature.
OVIDA platform providing access to specialized databases, journals, and medical and scientific resources.
Cochrane LibraryA high-quality source of evidence, including systematic reviews, meta-analyses, and the CENTRAL database of RCTs.
Web of ScienceAn interdisciplinary citation database enabling bibliometric analysis and literature exploration.
ScopusA large citation database supporting literature searching and analysis of research trends.
Health Source: Nursing EditionA database covering nursing, clinical care, rehabilitation, and health sciences.
AgricolaA database covering agriculture, veterinary medicine, environmental sciences, and related fields.
GreenFILEA database covering environmental research and the impact of human activity on ecosystems.
ERICeducational
ProQuest Dissertations & ThesesDissertations, academic theses, and other hard-to-access scholarly literature.
ClinicalTrials.govA registry of clinical studies containing information on study protocols and results.
EU Clinical Trials RegisterA European register of clinical trials of medicines conducted in EU/EEA countries.
WHO ICTRPA global platform providing access to clinical trial registries.
Search Strategy
TOOLSDESCRIPTION
PRESS ChecklistA checklist for assessing the quality of search strategies.
ISSG Search FiltersA collection of ready-made, tested search strategies.
Search BlocksA tool for creating and searching strategies for biomedical databases.
Systematic Review Development and Study Selection
TOOLSDESCRIPTION
JBI SUMARIA tool for creating and managing systematic reviews.
ZoteroA reference manager supporting the organization and deduplication of sources.
RayyanA tool for independent study screening (blinded review).
PRISMA Flow DiagramTemplates for visualizing the study selection process.
Protocol Registration
WEBSITEDESCRIPTION
PROSPEROInternational registry of systematic review protocols.
Cochrane ProtocolsRegistry of Cochrane review protocols.
OSF (Open Science Framework)An open platform for registering research protocols and projects.
Risk of Bias and Study Quality Assessment
TOOLSDESCRIPTION
RoB 2Cochrane tool for assessing the risk of bias in randomized controlled trials (RCTs).
ROBINS-ITool for assessing the risk of bias in non-randomized studies.
CASPChecklists for assessing the quality of qualitative studies.

Training Resources


Literature

Books available at the WMU Library

Open Access E-books


The WMU Library support

Department of the Scientific Information and Library Promotion invites the WMU community to take advantage of consultations on systematic reviews and literature reviews.

Consultations include support with:

Contact: informacja.naukowa@umw.edu.pl