Impact research and evaluation methods

Introduction

A short general introduction on this topic comes from the Johns Hopkins University Center for Communication Programs. "Impact research estimates the net effects of an intervention apart from other factors that might affect the outcome. Impact is best measured by the classical experimental design, with random assignment of subjects to treatment and control groups. Activities that allow for random assignment, such as interpersonal communication/counselling training and some types of community mobilisation are used when feasible. Activities reaching entire populations, such as mass media, cannot use this design. JHU/CCP generally use the quasi-experimental designs and research techniques listed below to evaluate these and other combined activities. A combination of these methods allows for the strongest possible causal inference about the effects of communication". They also give examples of some of their methods for impact evaluation.

Two major evaluation studies on hygiene promotion

Two major studies have been done on evaluations of effectiveness of health/hygiene promotion. John Hubley (2001) is slightly more positive than Cave and Curtis (1999).

Hubley's main conclusion from research he started in 1997 is that: 'Evidence exists for many health topics and methods that well designed health education/communication programmes can bring about changes in knowledge, attitudes, behaviour change and in some cases improvements in health". See next section.

Cave and Curtis, based on their review of the literature of 31 cases between 1987 to 1999, claim that: "although health education is promoted as a major factor in preventing ill health and disease, there is little evidence of its actual impact in terms of changing people's behaviour".

However, they raise an important question about whether health education is sustained for long enough periods. "Marketing specialists do not expect miracles in the way some health educators seem to. Take-off curves for some consumer durables in the US show how behaviour change can be long and slow. For example few colour TVs were sold between their introduction in 1954 and 1962, when sales went over the thousand and climbed steeply from there. Answering machines took for years to take off, as did electric shavers. Blenders took about 14 years for sales to go above 1000 per year. One reason why results haven been poor for most of the reported studies of health promotion may simply be because most interventions are too short and evaluations too early to show a clear impact".