Meta-Analysis (College Board AP® Psychology): Study Guide

Meta-analysis

• A meta-analysis is a quantitative research method that combines and statistically analyzes data from multiple published studies on the same topic

• Rather than conducting new research, the researchers use data that has already been collected and published by other researchers

  • This is called secondary data

• For example:

  • A meta-analysis of 150 studies investigating the relationship between sleep deprivation and academic performance

  • A meta-analysis of 80 studies examining the effectiveness of cognitive behavioral therapy (CBT) for treating depression

• By combining the findings of multiple studies, researchers can identify patterns and trends that would not be visible when analyzing a single study in isolation

How meta-analysis works

• Researchers identify all relevant published studies on a specific topic that meet pre-determined criteria

• The numerical findings from each study are extracted and combined using a statistical procedure

• The overall result is expressed as an effect size — a numerical value that indicates the strength of the relationship between two variables

  • A large effect size indicates a strong relationship between the variables across the studies analyzed

  • A small effect size indicates a weak relationship

• For example, an effect size of 0.8 for the relationship between physical exercise and reduced symptoms of depression would indicate a large, meaningful effect across the studies included

Meta-analysis and the evolution of scientific conclusions

• Meta-analysis plays a central role in how psychological conclusions evolve through peer review and replication:

  • Individual studies are peer-reviewed before publication — meaning other experts in the field have evaluated the methodology and findings for accuracy and validity

  • A meta-analysis draws on multiple peer-reviewed, replicated studies, meaning its conclusions are based on a much broader and more robust evidence base than any single study could provide

  • When multiple independent studies consistently produce similar findings, this strengthens confidence that the conclusion reflects a genuine psychological phenomenon rather than a one-off result

  • Conversely, if studies produce inconsistent findings, a meta-analysis can identify this variability and prompt further research to resolve the discrepancy

Evaluation of meta-analysis

Strengths

• Because the researchers are analyzing data collected by others, they cannot have influenced the outcome of the original studies — this reduces the risk of researcher bias and increases the reliability of the findings

• Combining data from a large number of studies increases statistical power — the ability to detect a true effect — making the conclusions more robust than those drawn from a single study

• The large and varied samples across multiple studies increase the generalizability of the findings to a wider population, strengthening external validity

Limitations

• Because researchers are relying on secondary data, they cannot be fully confident about the precision with which key variables were operationalized in the original studies — inconsistencies in how variables were defined and measured across studies can limit the reliability of the overall conclusions

• Researchers may struggle to access all relevant studies on a topic — studies that found no significant results are less likely to have been published, meaning the meta-analysis may overrepresent positive findings; this is known as publication bias and can distort the overall conclusions