Theoretical Issues: Validity, Reliability, Representativeness

Theoretical issues in research methodology are concerned with the fundamental question: what kind of knowledge does a particular research method produce, and how good is that knowledge? The three central theoretical concepts are validity, reliability, and representativeness. Alongside these, we consider the role of objectivity, the concept of triangulation, and the relationship between theory and method. Understanding these concepts is essential for evaluating any piece of sociological research and for constructing sophisticated exam answers.

---

Validity

Validity refers to the extent to which a research method measures what it claims to measure and produces a true, authentic picture of the social phenomenon being studied.

Key Definition: Validity — the degree to which a research method or instrument accurately measures or captures what it is intended to measure or capture. Valid data provides a genuine, truthful representation of social reality.

Types of Validity

Type	Definition	Example
Internal validity	Whether the research method genuinely captures the phenomenon it claims to study	Does a questionnaire about 'religious belief' actually measure religious belief, or does it measure something else (e.g. cultural conformity, social desirability)?
External validity	Whether the findings can be generalised beyond the specific study to other settings, populations, or times	Can findings from a study of a single school be applied to all schools in England?
Face validity	Whether the method appears, on the surface, to be measuring what it claims to	A reading test that involves reading passages has face validity as a measure of reading ability
Construct validity	Whether the method accurately operationalises the theoretical concept it claims to measure	Does measuring 'social class' by occupation capture the full complexity of the concept?
Ecological validity	Whether the findings reflect real-life behaviour and settings, rather than artificial research conditions	Laboratory experiments typically have low ecological validity because the setting is artificial

Factors That Affect Validity

Factor	Effect on Validity
Imposed frameworks	Pre-set questions and answer categories (in questionnaires and structured interviews) force respondents into the researcher's framework of meaning, potentially distorting their true views — reducing validity
Social desirability bias	Respondents may give answers they think are socially acceptable rather than truthful, producing data that does not reflect their genuine beliefs or behaviour
Hawthorne effect	People who know they are being studied may alter their behaviour, so the researcher observes a 'performance' rather than natural behaviour
Interviewer effect	The characteristics or behaviour of the interviewer may influence responses, producing data that reflects the interview situation rather than the respondent's true views
Researcher interpretation	In qualitative research, the researcher's interpretation of what they observe or hear may not accurately reflect the participant's intended meaning
Context	The setting in which data is collected (e.g. a formal interview room vs a participant's home) can affect how people respond

Methods and Validity

Method	Typical Level of Validity	Explanation
Questionnaires (closed)	Low	Pre-set categories impose the researcher's framework; social desirability bias; no opportunity to explore meanings
Structured interviews	Low-Medium	As above, though the interviewer can clarify questions
Unstructured interviews	High	Participants express themselves in their own words; the researcher can explore meanings in depth
Participant observation	High	The researcher observes actual behaviour in natural settings over extended periods
Experiments	Low ecological validity	Artificial settings do not reflect real-life conditions; demand characteristics may alter behaviour
Official statistics	Contested	Positivists regard them as valid measures; interpretivists argue they are social constructions

---

Reliability

Reliability refers to the consistency and replicability of a research method. A reliable method is one that, if repeated by a different researcher or at a different time, would produce the same or very similar results.

Key Definition: Reliability — the extent to which a research method produces consistent results when used repeatedly under the same conditions. A reliable method is one that can be replicated.

Types of Reliability

Type	Definition
Test-retest reliability	Whether repeating the same study with the same participants at a different time produces the same results
Inter-rater reliability	Whether different researchers using the same method on the same data produce the same results

Factors That Affect Reliability

Factor	Effect on Reliability
Standardisation	Highly standardised methods (questionnaires, structured interviews) produce consistent, comparable data that can be replicated — high reliability
Researcher involvement	The more the researcher is personally involved in data collection (e.g. in unstructured interviews or PO), the less replicable the study is — low reliability
Subjectivity	Qualitative methods involve subjective interpretation; different researchers may interpret the same data differently
Changing social context	Social life is constantly changing, so repeating a study at a different time may produce different results — not because the method is unreliable, but because the social world has changed
Small samples	Studies with small, purposive samples are harder to replicate because the specific participants cannot be replaced

Methods and Reliability

Method	Typical Level of Reliability	Explanation
Questionnaires	High	Standardised questions and format; easy to replicate
Structured interviews	High	Fixed interview schedule; each interview follows the same procedure
Unstructured interviews	Low	Each interview is unique; depends on the specific rapport and direction of conversation
Participant observation	Low	Each study is unique; depends on the researcher's relationship with the group, the specific setting, and the period of observation
Experiments	High	Controlled conditions and standardised procedures make replication straightforward
Official statistics	High	Standardised definitions and collection methods (though definitional changes over time can reduce comparability)

---

The Validity-Reliability Trade-Off

One of the most important insights in sociological methodology is that validity and reliability often pull in opposite directions:

High Reliability	High Validity
Standardised, structured methods	Flexible, open-ended methods
Pre-set questions and categories	Participants express themselves freely
Easy to replicate	Unique, non-replicable studies
Quantitative data	Qualitative data
Large samples	Small, in-depth studies
Positivist preference	Interpretivist preference

This means that researchers often face a trade-off: increasing the reliability of a method (by standardising it) may reduce its validity (by constraining participants' responses), and vice versa.

Exam Tip: The validity-reliability trade-off is one of the most powerful evaluative tools you can use in methods essays. If a question asks you to evaluate a specific method, always consider whether it prioritises validity or reliability, and explain the consequences of that choice.

---

Representativeness

Representativeness refers to the extent to which the sample or group studied is typical of the wider population, allowing the researcher to generalise their findings.

Key Definition: Representativeness — the degree to which a sample accurately reflects the characteristics of the target population. A representative sample enables the researcher to generalise findings from the sample to the population.

Factors That Affect Representativeness