36 Gordon Square
London WC1H 0PD

T +44 (0) 20 7958 8251

3ie-LIDC seminar: how to evaluate complex interventions?

Mark Petticrew from the London School of Hygiene and Tropical Medicine (LSHTM) shed light on the challenges involved in evaluating the impact of complex development interventions at the 3ie-LIDC Seminar 'What works in international development'.

He started by an overview of the history of systematic reviews. They date back more than 100 years, to early social sciences, and entail a systematic gathering of evidence. Attempts to move from simple to complex interventions initially met with resistance from systematic reviewers.

While best evidence is obtained from reviewing simple interventions, policy-makers are actually interested in complex ones. Key development challenges, such as inequalities or poor health, are complex phenomena with complex remedies. One needs to analyse the factors underlying them in order to understand their roots. Interventions aimed at addressing such complex phenomena are often non-linear.

A complex intervention can be defined as one whereby multiple components interact with one another and the whole is more than the sum of its parts. There is a number of behaviours involved and a number of variable outcomes. The intervention itself cannot be divided into a sequence of easily identifiable steps.

Complexity has consequences for the research question posed and getting the question ‘right’ constitutes a major difficulty related to identifying the population, the intervention itself, the level of outcomes, the context, how theory fits in, etc.

Petticrew gave urban regeneration as an example of a complex intervention. However, for evaluation purposes urban regeneration can be treated as a simple intervention, or a simple analysis may be applied to it. What makes evaluations of urban regeneration projects problematic are difficulties in establishing a baseline (as there rarely is a period in city development when nothing is happening), often there is no control group, and it easier to differentiate between primary and secondary outcomes.

The following steps are involved in evaluating complex interventions:
•    Setting the research question(s)
•    Deciding what sort of evidence is needed to answer the question(s)
•    Identifying the possible sources of complexity and mapping them to the sources of evidence
•    Reviewing the evidence

In conclusion, Petticrew quoted the Ockham Razor principle – the explanation should only be as complex as needed, which also applies to systematic reviews. The problem being complex does not imply that the analysis should be complex as well.

More about the 3ie-LIDC Seminar Series