Claims and Realities in HIV Programme Evaluation

According to this 2-page paper, HIV/AIDS interventions face particular challenges in meeting funding goals on one hand, and proving impacts on the other. In some instances, this may result in approaches where evaluation methods and findings are insufficiently critical. The paper explores the loveLife programme in South Africa, and suggests that research approaches and claims made by the programme have questionable validity, which limits ability of the programme to change and adapt.

This paper addresses the fact that HIV prevention is a complex process and that it is impossible to absolutely measure the impacts of specific HIV prevention interventions. This poses problems for programme evaluation. Programme funders and implementers are often closely involved in the design of monitoring and evaluation systems, and findings that suggest impacts are useful to securing ongoing funding. It therefore requires considerable discipline and critical capacity to ensure that findings are not skewed to support these long-term goals.

The document looks at the example of loveLife where claims to high levels of impact helped to foster buy-in amongst political, media and corporate elites in South Africa – many of whom are represented on the programme’s local advisory board. Internationally, the programme has been presented as an effective model for HIV prevention, and the research claims above have been widely repeated in conferences and reports. The claims also formed part of the programme’s proposal to the Global Fund for HIV/AIDS, TB and Malaria – eliciting an initial commitment of US$12 million.

According to the paper, research findings, particularly quantitative findings, are often accepted at face value, and when repeated in the media, in brochures, reports and websites, they have the capacity to appear rigorously grounded and 'true'. When they are presented in ways that circumvent peer review and commentary by other researchers in the field, they are able to avoid or limit critique. Additionally, when findings are promoted internationally, they are well beyond the critical voice of local researchers and others who may be aware of the programme’s limitations.

The paper suggests that some research claims from loveLife are not valid and have made it difficult to understand what the programme's actual impacts might be. According to the author, an approach based on critical evaluation would have value in that it would provide the capacity to adjust the intervention design as one proceeds – no model is perfect from the start. Claims that the model is working prevent any modification of design. The author concludes that in the short-term, claims about impact help to secure funding, and positive evaluation findings are no doubt pleasing to donors who make financial commitments. In the long-term, however, the paper states that this may breed cynicism of HIV prevention programmes as a whole.