Evaluating Methods in Context

This lesson brings together the knowledge developed across the course to provide a comprehensive framework for evaluating any research method when applied to the study of education. In the AQA Methods in Context question, you must demonstrate the ability to weigh up the strengths and limitations of a named method for investigating a specific educational topic. This requires more than listing advantages and disadvantages — it demands sustained, applied evaluation that considers practical, ethical, and theoretical issues in relation to both the method and the educational context described in the Item.

---

What Does 'Evaluation' Mean in Methods in Context?

Evaluation in the context of a Methods in Context answer means making judgements about the suitability of a method for researching a particular educational topic. This involves:

1. Identifying the strengths and limitations of the method
2. Applying these to the specific educational context described in the Item
3. Weighing up the relative importance of different issues
4. Reaching conclusions about the overall suitability of the method

Key Point: Evaluation is not simply listing points 'for' and 'against'. It involves analysis — explaining WHY a particular strength or limitation matters for this particular topic, and HOW it would affect the research in practice.

---

The PETT Framework Revisited

As established in Lesson 1, the PETT framework provides a systematic structure for evaluating methods in context:

Letter	Category	Focus
P	Practical issues	Can the research actually be done? Is it feasible in a school setting?
E	Ethical issues	Is the research morally acceptable? Does it protect participants' rights?
T	Theoretical issues	Does the method produce valid, reliable, and representative data?
T	Theoretical perspectives	Which sociological perspective would favour or criticise this method, and why?

However, a strong answer does not mechanically work through PETT as a checklist. Instead, it integrates these issues, showing how practical, ethical, and theoretical considerations interact and sometimes conflict.

---

Integrating Issues: How PETT Categories Overlap

In practice, practical, ethical, and theoretical issues are interconnected. Strong evaluation recognises these connections:

Example 1: Covert Observation in Schools

Issue	Analysis
Practical	Covert observation avoids the Hawthorne effect, as participants do not know they are being studied. The researcher can observe naturally occurring behaviour.
Ethical	However, covert observation involves deception and violates the principle of informed consent. This is particularly serious when children are the subjects of the research.
Theoretical	Covert observation may produce highly valid data (behaviour is natural), but the findings are shaped by the researcher's subjective interpretation, reducing reliability.
Integration	There is a direct tension between the practical and theoretical advantages of covert observation (natural behaviour, high validity) and its ethical unacceptability in educational research involving children. The ethical constraints effectively rule out covert observation in most educational contexts.

Example 2: Questionnaires with Young Children

Issue	Analysis
Practical	Young children may lack the literacy skills to read and understand questionnaire items, making the method impractical for primary school pupils.
Ethical	Children may not understand what they are consenting to; the concept of voluntary participation may be meaningless to very young pupils.
Theoretical	Even if children can complete the questionnaire, their responses may lack validity — they may interpret questions differently from the researcher, tick boxes randomly, or give responses influenced by what they think the adult wants to hear.
Integration	The practical limitations of using questionnaires with young children directly undermine both the ethical integrity and the theoretical value of the research. The method is unsuitable for this particular group.

---

Evaluative Techniques

Weighing Up

Not all strengths and limitations carry equal weight. A strong answer considers which issues are most significant for the specific research context:

• If the Item describes a highly sensitive topic (e.g. sexual abuse, racism), ethical issues may be more important than practical convenience.
• If the Item describes a large-scale pattern (e.g. national trends in achievement), representativeness and reliability may be more important than validity.
• If the Item describes a hidden process (e.g. teacher labelling, playground bullying), validity and access to genuine behaviour may be more important than reliability.

Comparing Methods

Although the question asks about a specific method, effective evaluation often involves implicit comparison with alternative methods:

• 'Questionnaires would produce reliable data, but they would lack the validity that could be achieved through unstructured interviews, which would allow pupils to explain their experiences in their own words.'
• 'While observation would reveal what actually happens in the classroom, it is more time-consuming and ethically complex than using existing school records.'

Considering the Research Topic

The suitability of a method depends heavily on what is being researched. The same method may be excellent for one educational topic and poor for another: