Well-Designed Experiments (College Board AP® Statistics): Revision Note
Syllabus Edition
First teaching 2026
First exams 2027
Confounding variables
What are confounding variables?
A confounding variable is a variable that
you are not interested in
but that can affect the results of your experiment
e.g. the levels of background noise when trying to conduct a memory test
As the explanatory variable changes, the confounding variable also changes
These two changes both influence the response variable
This makes it hard to draw conclusions
For example, it may look like increasing coffee drinking increases rates of heart disease
but actually increasing coffee drinking increases the tendency to smoke, which increases rates of heart disease
The level of smoking is the confounding variable
What should I do if there are confounding variables?
In an experiment, it is important to
identify possible confounding variables before beginning an experiment
control (minimize or eliminate) the effect of any confounding variables
e.g. conduct memory tests in a quiet room
In an observational study, you cannot control any confounding variables
This makes it harder to know what is causing what
Worked Example
A school district wants to evaluate the effectiveness of a new voluntary after-school tutoring program in improving standardized math test scores. Students who opt into the tutoring program attend sessions twice a week. At the end of the year, the district compares the test scores of students who participated in the tutoring program to the scores of students who did not participate. The district finds that the students in the tutoring program scored significantly higher on the standardized test and concludes that the program caused the improved scores.
Which of the following is a confounding variable in this study?
(A) The type of after-school program (tutoring or no tutoring)
(B) The standardized math test scores at the end of the year
(C) The students' choice of whether or not to participate in the tutoring program
(D) The fact that both groups of students took the exact same standardized math test
Answer:
(A) is the explanatory variable (or factor) whose levels are being compared
(B) is the response variable, which is the outcome measured after the treatment has been administered
(D) is a form of direct control, keeping the measurement tool the same for all students to reduce extraneous sources of variation
The students' choice to participate means the groups were not randomly assigned
Students who voluntarily opt into extra tutoring may already possess a higher level of academic motivation or dedication than those who do not
This underlying motivation is associated with both the explanatory variable (the choice to join the program) and the response variable (higher test scores), making it impossible to tell if the tutoring program or the students' pre-existing drive caused the higher scores
The correct answer is C
Well-designed experiments
What is a well-designed experiment?
A well-designed experiment consists of the following:
At least two treatment groups (comparing one group to another group)
A control group counts as a treatment group
Treatment groups are formed by randomly assigning treatments to the experimental units
This keeps the groups as similar as possible before the experiment
This makes it easier to distinguish responses to treatments only
This process is called randomization
Treatment groups have more than one experimental unit each
This reduces the effects of any natural variations
This is called replication (the more the better!)
Confounding variables are identified and controlled
This ensures they stay the same across all treatment groups
Examiner Tips and Tricks
In this course, replication does not mean repeating the experiment. Replication means multiple experimental units per treatment group.
Statistically significant experiment results
What are statistically significant experiment results?
The results of an experiment are called statistically significant if the changes in the response variable (or the differences between treatment groups) are so large that they are unlikely to be down to chance
It suggests that there is a relationship between the treatment and the response
When can I use the word “cause” in my conclusions?
You can conclude that the treatment causes the response if the following two conditions are met:
Treatments were randomly assigned to experimental units
The experimental results are statistically significant
When can I generalize the results from an experiment to the population?
You can generalize the results from an experiment to a population if
the sample of experimental units used in the experiment was randomly selected from the population
random selection reduces bias in the sample and makes it more representative of the population
Examiner Tips and Tricks
Do not confuse the process of randomly selecting experimental units from a population to use in your experiment with the subsequent process of randomly assigning your selected experimental units the different treatments!
Unlock more, it's free!
Join the 100,000+ Students that ❤️ Save My Exams
the (exam) results speak for themselves:
Was this revision note helpful?
