Reliability of Qualitative & Quantitative Methods (AQA GCSE Psychology): Revision Note

Reliability is a measure of consistency. If a measurement can be repeated and produces the same result, it is described as reliable

• e.g. if a set of weighing scales is working properly, it would give the same weight for the same (unchanged) object every time it was measured

Psychologists use methods such as questionnaires, observations, interviews and case studies to 'measure' people instead

• For the findings to mean anything, these methods need to be consistent

Quantitative methods are those which generate numerical data, for example:

• Lab experiments

• Closed questions on a questionnaire

• Observations using a tally chart to record the frequency of specific behaviours

Generally, methods that collect quantitative data tend to be the most reliable

Lab experiments are considered the most reliable method in psychology because:

• The researcher can control and standardise the procedure, so it can be easily replicated in exactly the same way for every participant

• They generate quantitative data, which is straightforward to compare and analyse for consistency

Field experiments involve less control over extraneous variables in the real-life setting

• This makes exact replication harder so they tend to be less reliable than lab experiments

Natural experiments use a naturally occurring IV that the researcher cannot control

• This makes them harder to replicate exactly, and so less reliable than lab experiments

Questionnaires are a relatively reliable self-report method because they:

• tend to use closed questions, which generate quantitative data that is easy to compare between testing sessions

• use a standardised set of questions, given in the same way to every participant

• are likely to show consistency, as people are unlikely to radically change their answers from one session to the next

It is easier to achieve reliability with closed questions, since the choice of answers is fixed, than with open questions, where a person might phrase their answer slightly differently each time even if their underlying view hasn't changed

The reliability of a questionnaire can be checked using two methods:

• Test-retest (a check of external reliability): the same participants complete the same questionnaire on two separate occasions (e.g. with a gap of several months)

  • If each participant gives similar answers both times, external reliability is established

• Split-half (a check of internal reliability): the researcher splits the questionnaire in half and compares responses to the first half with responses to the second half

  • If the two halves produce similar results, internal reliability is established

Before an observation begins, researchers agree in advance which specific behaviours will be recorded, so all observers are working from the same behaviour categories

Inter-observer reliability is the level of consistency between two or more trained observers conducting the same observation:

• All observers agree on the behaviour categories and how to record them before the observation begins

• Each observer records independently, to avoid one observer being influenced by another

• Afterwards, the two independent data sets (often tally charts) are compared and a correlation is calculated between them

• A strong positive correlation shows good inter-observer reliability, and that the behaviour categories worked reliably

Good inter-observer reliability means there is less chance that observer bias has affected the data

Qualitative methods are generally regarded as less reliable than quantitative ones

Methods such as unstructured interviews and case studies are difficult to repeat in exactly the same way:

• In an unstructured interview, interviewees are encouraged to talk freely about their thoughts and feelings

  • Even with the same person, the interview is unlikely to unfold the same way twice, so there is a lack of consistency (reliability)

• Even a structured interview, where the questions are the same every time, may still be delivered differently by different interviewers, whose own style can prompt different replies

Qualitative data (such as interview transcripts or case study material) is also much more open to interpretation than numerical data

Two different researchers might draw different conclusions from the same material, meaning the analysis itself may lack consistency (reliability)