Pilot Studies & the Aims of Piloting (College Board AP® Psychology): Revision Note

Pilot studies

• A pilot study is a small-scale trial run before the main study to test some or all aspects of the proposed procedure

  • It acts as a 'dress rehearsal' — identifying problems early so they can be fixed before the full study is conducted

• A pilot study can identify:

  • flaws in operational definitions — are the IV and DV defined and measured clearly enough to produce meaningful, replicable data?

  • confounding variables — does the procedure inadvertently introduce variables that could distort the results?

  • feasibility issues — is the proposed design practically achievable?

  • issues with the hypothesis — are the measures sensitive enough to actually test whether the hypothesis is falsifiable?

  • sampling issues — is the sample likely to be representative enough to allow generalization of results?

  • reliability and validity — is the measure consistent, and is the study actually testing what it sets out to test?

  • ethical issues — does the procedure cause any unexpected distress, risk, or discomfort to participants?

• After a pilot study, if problems are identified, the researcher makes adjustments, for example:

  • Revising unclear instructions and definitions

  • Changing the research design if the original is flawed (e.g., switching from repeated measures to matched pairs to reduce order effects)

  • Refining the sample if generalizability is a concern

  • Any unexpected distress caused by the procedure can be flagged and addressed prior to IRB/institutional review approval

• If changes are made, a further pilot study should be run to test the revised procedure before the main study begins