Evaluating & Improving Psychological Research (College Board AP® Psychology): Flashcards

Reliability can be improved by using more than one observer and establishing inter-rater reliability before the main observation, and by operationally defining behavioral categories so they are mutually exclusive.

Validity is the extent to which a study measures what it claims to measure and whether its findings accurately reflect real thoughts, feelings, and behavior.

Internal validity is about whether changes in the DV were caused by the IV rather than confounding variables, while external validity is about whether the findings can be generalized beyond the research setting.

False.

Ecological validity depends on whether the task reflects genuine real-world behavior, not simply where it takes place, so a natural setting with an artificial task can still have low ecological validity.

Face validity is the extent to which a test appears, on the surface, to measure what it claims to measure, based on the content of its questions.

Concurrent validity is established when scores on a new test show a strong positive correlation with scores on an already established test of the same construct.

Internal validity is lowered when confounding variables have not been controlled, when experimenter bias or demand characteristics influence behavior, or when the IV and DV are not clearly operationally defined.

Predictive validity is the degree to which a test score accurately predicts future performance on a relevant outcome, such as high IQ scorers going on to perform well academically.

A double-blind procedure controls for both demand characteristics and experimenter bias by ensuring that neither the participant nor the researcher knows which condition the participant is in.

A study has high population validity when its sample is representative of the wider target group it claims to apply to.

Construct validity is the degree to which a test actually measures the theoretical construct it claims to measure, shown when results align with what the theory predicts.

Validity can be improved by using covert methods where ethically justifiable, so that participants display natural behavior, and by ensuring behavioral categories are operationally defined, observable, and mutually exclusive.

Peer review is the independent assessment of research conducted by other experts in the field before it is published.

Peer reviews are conducted independently and anonymously, so reviewers do not know who conducted the research and researchers do not know who is reviewing their work.

False.

This outcome means the researcher is invited to make significant changes and resubmit for review; it is not publication approval.

Replication is the process of repeating a study using the same methodology to test whether the original findings can be consistently reproduced.

One of the four possible peer review outcomes is "accept with revisions", meaning publication is conditional on specific improvements being made.

Peer review aims to check the validity, reliability, significance, and originality of research, and to suggest revisions where necessary before publication.

A replication failure is when a repeat of an original study fails to reproduce its findings, which prompts a re-evaluation of the original methodology and conclusions.

False.

Even a peer-reviewed study cannot establish a finding as reliable or valid on its own; independent replication is needed to build confidence in the conclusion.

A finding that replicates consistently across diverse samples and settings has greater external validity than one shown in a single study.

Peer review acts as a quality control filter, ensuring that only credible, valid, and significant research enters the scientific literature.

The abstract is a brief summary of the entire research report, giving an overview of the aim, hypothesis, method, key results, and main conclusions.

The introduction provides the theoretical and empirical context for the study, including relevant theories, previous research, the aim, and both the experimental and null hypotheses.

False.

The method section describes exactly how the study was conducted in enough detail for another researcher to replicate it; the results are presented in the results section.

The discussion section interprets the results in the context of the study's aim and the broader theoretical and empirical literature, and evaluates the methodology.

The descriptive statistics in a results section include measures of central tendency, such as the mean, median, or mode of scores per condition.

The Participants part reports the total number of participants, relevant demographic details such as age and gender, and the sampling method used with its justification.

The method section describes exactly how the study was conducted, with enough detail about participants, design, materials, and procedure for another researcher to replicate it precisely.

True.

Reading the abstract first lets reviewers quickly judge whether a study is relevant and worth scrutinizing in detail; a flawed abstract prompts closer scrutiny of the full report.

Inferential statistics in a results section indicate whether the findings are statistically significant and whether the null hypothesis is rejected.

A clearly written method section allows independent researchers to repeat the study under the same conditions and test whether the original findings hold up.

A meta-analysis is a quantitative research method that statistically combines and analyzes the numerical findings of multiple published studies on the same topic.

Pre-determined criteria are used to identify all relevant published studies on a specific topic for inclusion in the meta-analysis.

False.

A meta-analysis is not a literature review; it statistically combines the numerical findings of multiple studies to produce a new overall conclusion.

Publication bias is the tendency for studies with no significant results to go unpublished, meaning a meta-analysis may overrepresent positive findings and distort its conclusions.

Rather than collecting new data, meta-analysis researchers use secondary data that has already been collected and published by other researchers.

Because the researchers are analyzing data collected by others, they cannot have influenced the outcome of the original studies, which reduces the risk of researcher bias.

Effect size is a numerical value that indicates the strength of the relationship between two variables across the studies included in a meta-analysis.

Combining data from a large number of studies increases statistical power, making a meta-analysis's conclusions more robust than those of a single study.

Because the researchers rely on data collected by others, inconsistencies in how key variables were defined and measured across the original studies can limit the reliability of the overall conclusions.

Combining large, varied samples across multiple studies increases generalizability, making the findings more likely to apply to a wider population and strengthening external validity.