Validity (College Board AP® Psychology): Revision Note

Types of validity

• Validity refers to the extent to which a study measures what it claims to measure and whether its findings accurately reflect real thoughts, feelings, and behavior

• A study is valid if:

  • the results are due to the manipulation of the IV rather than the influence of confounding variables

  • the findings can be generalized beyond the research setting to the wider population

  • the measurement instruments used actually measure what they are intended to measure

• Validity is essential to the scientific process - findings that lack validity:

  • cannot be meaningfully interpreted

  • will not survive peer review

  • cannot contribute to the evolution of psychological knowledge

Types of Validity

Internal validity

• Internal validity measures the extent to which changes in the DV are due to the manipulation of the IV rather than the influence of confounding variables

• A study has high internal validity when

  • confounding variables have been identified and controlled

  • random assignment has been used to distribute participant variables evenly across conditions

  • single-blind or double-blind procedures have been used to control for demand characteristics and experimenter bias

  • the procedure is fully standardized across all conditions

• A study has low internal validity when

  • confounding variables have not been controlled — alternative explanations for the results exist

  • experimenter bias or demand characteristics have influenced participant behavior

  • The IV and DV are not clearly operationally defined

External validity

• External validity measures the extent to which the results of a study can be generalized beyond the research setting

• There are two key aspects of external validity:

  • Ecological validity

  • Population validity

Ecological validity

• Ecological validity refers to the extent to which the findings reflect real-world behavior

  • Does the task participants are given feel natural and reflect genuine everyday experience?

• If it has high ecological validity, the task closely mirrors a real-life situation

  • This means the findings are more likely to reflect how people actually think and behave outside of the research setting

• If the task has low ecological validity, the task is artificial or contrived

  • This means the participants may behave differently than they would in real life, limiting the generalizability of the findings

• For example:

  • A lab experiment asking participants to memorize a list of random words has low ecological validity — this does not reflect how memory is used in everyday life

  • A field experiment observing helping behavior on a real subway train has higher ecological validity — the situation closely mirrors a genuine real-world scenario

Population validity

• Population validity refers to the extent to which findings can be generalized from the sample to the wider target population

  • A study has high population validity when the sample is representative of the target population

  • A study has low population validity when the sample is biased

    • E.g. a study conducted exclusively on college students cannot be confidently generalized to the broader population

Measuring validity

• There are different ways of measuring validity depending on the type of research and the measurement instrument used

Face validity

• Face validity measures whether a test appears to measure what it claims to measure on the surface

  • E.g. does this anxiety scale look like it is measuring anxiety, based on the content of its questions?

• Face validity is the most basic form of validity assessment

  • It does not guarantee that the measure is actually valid, only that it appears to be

Concurrent validity

• Concurrent validity measures how closely a new test agrees with an already established test of the same construct

• Participants complete both the new test and the established test — if the scores show a strong positive correlation, this is evidence of concurrent validity

  • E.g.scores on a new IQ test are compared with scores on an established, validated IQ test — a correlation of +0.8 or above would indicate good concurrent validity

Improving validity

• If validity is found to be low, the researcher must take steps to improve it before the study is conducted or repeated

• The appropriate strategy depends on the research method being used

Lab and field experiments

• Use clearly operationally defined IV and DV

  • Vague definitions allow for alternative interpretations of the results and reduce internal validity

• Use a control group as a baseline for comparison

  • Without a control group it is impossible to determine whether changes in the DV were caused by the IV

• Use random assignment to distribute participant variables evenly across conditions

  • This reduces the risk that pre-existing differences between participants act as confounding variables

• Use single-blind procedures to control for demand characteristics

  • Participants who do not know which condition they are in cannot alter their behavior based on perceived expectations

• Use double-blind procedures to control for both demand characteristics and experimenter bias

  • Neither the participant nor the researcher knows which condition the participant is in, preventing either from influencing the results

Observational studies

• Use covert methods where ethically justifiable

  • Participants who are unaware they are being observed are more likely to display natural, unforced behavior, increasing ecological validity

• Ensure behavioral categories are clearly operationally defined, observable, and mutually exclusive

  • Ambiguous categories introduce subjectivity and reduce the validity of the findings

Surveys and interviews

• Ensure questions are clearly worded, neutral, and free from leading questions

  • Poorly worded questions introduce self-report bias and social desirability bias, reducing the validity of responses

• Use closed questions or Likert scale items where appropriate to reduce ambiguity in responses

Validity & the evolution of scientific conclusions

• Validity is central to how psychological conclusions evolve through peer review and replication:

  • During peer review, experts evaluate whether a study's methodology is sufficiently valid to support the conclusions drawn

    • Studies with low internal or external validity are likely to be challenged or rejected

  • When a study is replicated and produces consistent findings across different samples and settings, this strengthens both its internal and external validity

  • Findings that lack ecological validity or population validity are less likely to be accepted as meaningful contributions to psychological knowledge

    • This is because they cannot be generalized beyond the specific context in which they were obtained

  • Over time, as studies are replicated, peer reviewed, and refined, invalid measures are identified and replaced with more valid alternatives

    • This is how psychological measurement improves