Surveys (College Board AP® Psychology): Study Guide

Surveys

• A survey is a self-report method in which participants answer a set of questions designed to collect their thoughts, feelings, attitudes, and opinions

• Surveys may be used to:

  • measure psychological constructs such as anxiety, depression, or empathy

  • understand attitudes toward social issues such as immigration, social media use, or political preferences

  • collect large amounts of data from a wide sample quickly and efficiently

• Surveys are a non-experimental methodology — there is no manipulation of an IV

• Surveys can consist of closed questions, open questions, or a combination of both

Closed questions and quantitative data

• A closed question offers participants a limited set of response options, e.g.

  • "Do you feel anxious in social situations? Yes / No"

  • "Which of the following best describes your mood today?

    • a) Happy b) Neutral c) Sad d) Anxious"

• Closed questions generate quantitative data — numerical scores that can be easily analyzed and compared across participants

• Two common types of closed question used in surveys:

  • Likert scale questions — participants indicate the degree to which they agree with a statement, e.g.

    • "I feel anxious in social situations"

    • Strongly Agree / Agree / Neutral / Disagree / Strongly Disagree

  • Rating scale questions — participants select a numerical value to indicate the strength of their response, e.g.:

    • "How anxious do you feel right now?"

    • 1 (Not at all anxious) — 10 (Extremely anxious)

Evaluation of closed questions

Strengths

• Closed questions generate quantitative data that is straightforward to analyze, compare across participants, and represent graphically

• Standardized response options make surveys easy to replicate, which supports reliability

Limitations

• Quantitative data sacrifices depth and detail

  • Closed questions can show what participants think or feel but not why, which limits the validity of the findings

• Fixed response options may not accurately capture a participant's true experience if none of the options reflects their actual response

Open questions and qualitative data

• An open question allows participants to respond freely in their own words, e.g.:

  • "Describe a time when you felt anxious in a social situation"

  • "How do you think stress affects your daily life?"

• Open questions generate qualitative data — rich, detailed, descriptive responses that capture the complexity of individual experience

Evaluation of open questions

Strengths

• Closed questions produce detailed, explanatory data that gives insight into the reasons behind participants' attitudes and behavior

  • This means data is high in explanatory power and external validity

• Closed questions allow participants to express their experience in their own words, which may more accurately reflect their true thoughts and feelings

  • This increased the validity of the findings

Limitations

• Qualitative data is difficult to analyze objectively

  • The researcher must interpret meaning from responses, which introduces subjectivity and reduces reliability

• Responses cannot easily be compared across participants or represented statistically

Survey wording

• Survey wording is one of the most important factors affecting the validity of survey data

• Poorly worded questions can lead to two key forms of bias:

  • self-report bias

  • social desirability bias

Self-report bias

• Self-report bias occurs when the wording of a survey question influences how participants respond, leading them to give inaccurate answers

• This can occur when questions are leading, i.e. where the wording of the question steers participants toward a particular answer, e.g.

  • "Don't you agree that social media is damaging young people's mental health?"

• The leading question contains an emotional element and implies an expected answer

• A suitable question that would be neutral and unbiased is:

  • "To what extent do you agree that social media affects young people's mental health?"

• Self-report bias can also occur when questions are ambiguous or use technical jargon that participants do not understand, causing them to guess or respond randomly

Social desirability bias

• Social desirability bias occurs when participants respond in ways they believe are socially acceptable or favorable rather than truthfully

• This is particularly common in surveys on sensitive topics such as prejudice, substance use, aggression, or mental health

  • E.g. participants may underreport alcohol consumption or overreport charitable behavior to present themselves in a more positive light

• Social desirability bias threatens the internal validity of the survey, as the responses do not accurately reflect participants' true attitudes or behavior

Qualitative vs. quantitative measurement instruments

• It is important to distinguish between qualitative and quantitative measurement instruments in survey research:

Evaluation of surveys

Strengths

• Surveys are a quick, cost-effective way of collecting data from large samples

  • This is a strength as large samples produce more reliable results as anomalous responses are balanced by the overall trend of the data

• The use of standardized questions means surveys can be replicated easily

  • This allows reliability to be checked over time

• Surveys can reach geographically diverse participants through online platforms

  • This increases the representativeness of the sample and the generalizability of the findings

Limitations

• Social desirability bias is a significant limitation of surveys

  • Participants may not respond truthfully, particularly on sensitive topics, which reduces the validity of the findings

• Survey wording can introduce self-report bias

  • Leading questions, ambiguous wording, or technical jargon can distort participants' responses and undermine the validity of the data collected