36 Gordon Square
London WC1H 0PD

T +44 (0) 20 7958 8251

3ie-LIDC Seminar Series - “Systematic Reviews of Complex Interventions”

Wednesday, March 21, 2012 - 17:15 to 17:50
Time: 5:15 - 6:45 PM
Venue: London International Development Centre, 36 Gordon Square, Lower Meeting Room, London WC1H 0PD

Speaker: Mark Petticrew
Discussant: Sandy Oliver
Chair: Hugh Waddington

Mark Petticrew from the London School of Hygiene and Tropical Medicine (LSHTM) shed light on the challenges involved in evaluating the impact of complex development interventions at the 3ie-LIDC Seminar 'What works in international development'.

He started by an overview of the history of systematic reviews. They date back more than 100 years, to early social sciences, and entail a systematic gathering of evidence. Attempts to move from simple to complex interventions initially met with resistance from systematic reviewers.

While best evidence is obtained from reviewing simple interventions, policy-makers are actually interested in complex ones. Key development challenges, such as inequalities or poor health, are complex phenomena with complex remedies. One needs to analyse the factors underlying them in order to understand their roots. Interventions aimed at addressing such complex phenomena are often non-linear.

A complex intervention can be defined as one whereby multiple components interact with one another and the whole is more than the sum of its parts. There is a number of behaviours involved and a number of variable outcomes. The intervention itself cannot be divided into a sequence of easily identifiable steps.

Complexity has consequences for the research question posed and getting the question ‘right’ constitutes a major difficulty related to identifying the population, the intervention itself, the level of outcomes, the context, how theory fits in, etc.

Petticrew gave urban regeneration as an example of a complex intervention. However, for evaluation purposes urban regeneration can be treated as a simple intervention, or a simple analysis may be applied to it. What makes evaluations of urban regeneration projects problematic are difficulties in establishing a baseline (as there rarely is a period in city development when nothing is happening), often there is no control group, and it easier to differentiate between primary and secondary outcomes.

The following steps are involved in evaluating complex interventions:
•    Setting the research question(s)
•    Deciding what sort of evidence is needed to answer the question(s)
•    Identifying the possible sources of complexity and mapping them to the sources of evidence
•    Reviewing the evidence

In conclusion, Petticrew quoted the Ockham Razor principle – the explanation should only be as complex as needed, which also applies to systematic reviews. The problem being complex does not imply that the analysis should be complex as well.


About the speakers:
Professor Mark Petticrew , Professor of Public Health Evaluation in the Department of Social and Environmental Health Research at the London School of Hygiene and Tropical Medicine.
He is Director of the Public Health Research Consortium (PHRC), and is also involved in the Policy Innovation Research Unit (PIRU), both of which are funded by the Department of Health Policy Research Programme.

Mark is editor of the new Cochrane Public Health Review Group, and is closely involved in the Cochrane/Campbell Health Equity Field. He is also an Honorary Professorial Fellow at the University of Melbourne School of Population Health: http://www.sph.unimelb.edu.au/, and an Honorary Researcher at the MRC Social and Public Health Sciences Unit in Glasgow.
Professor Sandy Oliver, Deputy Director, Social Science Research Unit and the EPPI-Centre, Institute of Education. She is a Professor of Public Policy and Deputy Director at the Social Science Research Unit and the EPPI-Centre at the Institute of Education (University of London). Her research interests centre on participatory approaches to doing and using research. She leads the EPPI-Centre’s support for four southern hemisphere review centres funded by the Alliance for Health Policy and Systems Research (WHO), for 20 DFID commissioned reviews and for review teams funded by 3ie, AusAID and DFID. She is an editor of the Cochrane Consumers and Communication Review Group, and a member of the WHO Expert Advisory Panel on Clinical Practice Guidelines and Research Methods and Ethics, and the international Task Force on Guidelines for Health Systems Strengthening Group.

Hugh Waddington, Senior Evaluation Officer, 3ie, is an Economist by training and manages 3ie’s Systematic Reviews Programme and the Campbell Collaboration International Development Coordinating Group. Before joining 3ie, Hugh was employed as an Overseas Development Institute (ODI) Fellow in the Ministry of Finance of the Government of Rwanda, where he worked on the elaboration of Rwanda’s Economic Development and Poverty Reduction Strategy (PRSP2). He has previously worked with the Economist Intelligence Unit, UK National Audit Office, Save the Children UK and the World Bank's Operations Evaluation Department. Hugh holds an MA in Development Economics from the University of Sussex.

More about the 3ie-LIDC Seminar Series