Democracy, Governance and Randomised Media Assistance

"What constitutes good evidence is currently subject to intense debate within the international development community. There is particular interest in making greater use of experimental and quasi-experimental research methods to assess the effect and impact of development interventions."

From the introduction: "This paper investigates the potential for field experiments to contribute to our understanding of the intersection between media, democracy and governance [DG]. It reviews the small body of pioneering field experiments on media and political development, and it outlines some of the obstacles to conducting and learning from field experiments in this sector. Finally, the paper suggests opportunities for building a body of experimental evidence that can inform DG interventions in the media assistance sector." [Footnotes are removed by the editor throughout.]

To address the fact that media assistance in developing countries is growing and some of it is or could be based upon research results, the paper looks at opportunities and challenges of randomised field experiments that address the use of media to achieve DG outcomes. "The first section describes the growth of DG assistance and the increasing interest in DG field experiments. The second section describes the domain of media assistance targeted at DG outcomes. The third provides an overview of current experimental and quasi-experimental studies on the intersection between media, democracy and governance. The fourth and fifth sections describe some of the challenges to successfully employing field experiments to inform media assistance programmes. The final section concludes by arguing for the benefits of practitioner-academic collaborations that provide experimental evidence about the influences on as well as the effects of media content related to democracy and governance."

Among research results, there are those that show positive results, such as a pre-election initiative in India that provided one experimental group with free newspapers covering the performance of incumbent legislators and the qualifications of candidates. The initiative showed results of increased turnout, higher preference for better-performing candidates (as presented in the newspapers), and reduced cash-based vote-buying. In contrast, a legislative transparency project in a Vietnam newspaper drew much consumer attention but began to deter legislators from breaking with regime norms due to increased scrutiny. Conflict-affected areas studied for media impacts are also described with diverse and sometime unexpected results; for example, results from research on partisan media broadcasts showed that some of the broadcasts caused discord, while others heightened nationalism while reducing factionalising tribal tension and discontent.

The challenges section focuses on: 1) level of intervention; 2) complexity of intervention; and 3) research planning under ambiguity. It describes varying levels of penetration of interventions and the need for research innovation to give meaningful results. "For example, passage of a freedom of information act may theoretically benefit all journalists within a country, but only those journalists who are knowledgeable about the procedures for filing requests will be able to realise these benefits... ", thus requiring care in randomised sampling. Because multi-faceted, complex approaches to programming pose inter-related challenges to hypothesis testing, often separating strands of media intervention and testing each strand yields clearer results. Programme flexibility in which decisions are made to deviate from initial goals and design can cause a "disjuncture between the optimal research design and baseline data collected...."

The research highlights three main opportunities within the challenges of evaluating media interventions:

1.  Because development programmes, including media development, are often conducted in areas of media scarcity, there may be "organic convergence of research attention to and programming involvement in certain types of environments." Researchers may be able to distinguish the effects of media from what led individuals to consume the media in the first place and to develop robust estimates of media effects, for example: comparing those with and without access to the media under investigation. In addition, media may be consumed publicly because individuals have less personal access to media; for example: Radio in taxis is controlled by drivers so that the effects of partisan media choice on riders can be measured and the common practice of listening to radio in groups can be used to "randomise exposure to study the effect of programmes to reduce prejudice and increase democratic attitudes."
2.  "[M]edia assistance programmes are often based on assumptions about how media affects democracy and governance. Field experiments seem to be especially well suited to test assumptions about media effects and thus provoke greater reflection about programme goals and theories of change." For example, the assumption that reporting on the performance of government officials increases transparency is challenged by a study that suggests that it led to decreased legislative independence.  Further, the finding that talk shows may increase intergroup hostility and another that suggests that discussion about women’s role in politics made individuals less favourable towards women’s inclusion "highlight the importance of testing assumptions about media effects."
3. "[Q]uasi-experimental and qualitative studies suggest that institutions affect media quality and ultimately democratic development. As yet, field experiments have not been conducted on the factors that influence media content and reach.  Creative thinking and collaborative efforts between researchers and practitioners could yield robust experimental evidence testing extant theories about how media are shaped by legal, regulatory economic and political environments." Other evaluation techniques used to study the effect of interventions on media content, institutions, and practices united with current research practices might include, for example, randomising training programmes and comparing articles or broadcasts produced by trained and untrained media professionals. In addition, "[e]xperimental evaluations could also be designed that encourage uptake of practices, adoption of norms, implementation of laws and enforcement of penalties in treatment groups but not in comparison groups." The document concludes that "[f]ield experiments would seem to add some value within a multi-method approach to evaluating media for DG interventions, although the ultimate contribution of the method depends on the ability of researchers to address creatively the distinctive advantages and challenges of the methodology."

Click here to read this 40-page research brief in PDF format.

Editor's note: A related research briefing - "What Can We Learn from Field Experiments on Media, Communication and Governance?" [PDF] - provides a "user friendly" resource to explain the process that informed the original research, exploring topics such as "what is a field experiment?" and "what is a quasi-experimental study?". It also summarises conclusions, key findings, and implications.