Strengthening Accountability Through Media in Afghanistan: Final Evaluation

"This project aimed to enable more accountable, peaceful and inclusive state-society relations in Afghanistan, by providing a platform for people to question their leaders and which built the profile and participation of women in public dialogue and debate. To support this, it also sought to raise the production skills and standards of RTA staff to produce high-quality coverage on key governance issues."

This BBC Media Action evaluation analyses its work with state broadcaster Radio Television Afghanistan (RTA) and the BBC Afghan Service to co-produce the TV and radio national debate programme Open Jirga (Open Assembly) and support the Afghan Education Production Organization (AEPO) to incorporate governance issues into its longstanding radio drama New Home, New Life. 

"In 2012, BBC Media Action developed an accountability conceptual model that posited that media can influence a range of individual and collective drivers of accountability that, in turn, reinforce each other. The individual drivers in the conceptual model have been distilled into five key constructs that are operationalised and measured in BBC Media Action’s research. These are: political knowledge, political participation, discursive participation (discussion), [and] interest." Between 2012 and 2017, the Open Jirga debate format was developed to allow Afghans to interact directly with officials. Ninety-three debates were broadcast. Capacity strengthening and radio drama were part of the project activities as well. The project’s theory of change, illustrated on page 13, figure 2 of the document, lists priority governance challenges and media and communication challenges, the five objectives – from the key constructs above, the proposed cumulative outcome, and the impact - "[m]ore accountable, peaceful and inclusive state-society relations in Afghanistan". 

Research methodology is described as mixed and on-going from the start of the project. "In total, over the course of the project, BBC Media Action undertook two nationally representative quantitative surveys [2013, 2014] and six qualitative studies (including an evaluation of the capacity strengthening with RTA)." The qualitative surveys were aimed to help shape the project and explore more nuanced impact on the audiences, as well as how to shape capacity building. AEPO did a qualitative survey of audiences in 2016 involving focus group discussions (FGDs) and individual interviews (IDIs). The evaluation synthesises cumulative findings.

"Findings showed that BBC Media Action's training and mentoring of RTA was largely successful. It was also found that more than 84% of those that tuned into Open Jirga and/or New Home, New Life reported that the programmes had increased their knowledge of key governance issues." The audiences (larger for the radio drama based upon a longer history of audience loyalty) shows statistical success in reaching women (close to half the audience) and the middle economic sector but was challenged to reach the less educated population sector.

The following factors were key to engagement with programmes: 

• the debate format - participation of high-ranking officials, topic selection, constructive dialogue, diversity and participation of studio audience, the presenter, bilingual programming
• the drama format - relevant characters and storylines
• addressing key governance and accountability issues to build knowledge of governance issues and role-model problem-solving behaviour at the family and community level.

The following key insights were drawn from the cumulative responses:

• 84% reported increased knowledge of key governance issues,
• But audiences also reported less confidence about being able to address issues with high-level officials outside of the programme format.
• Radio drama listeners realised a responsibility to check that public works were carried out to a satisfactory standard and to budget.
• Both programmes informed audiences about issues affecting women and rights of women.
• The debate format raised the profile of women in public debate, but they continued to be barred from accessing public official except through men.
• Audiences did take local actions.

The evaluation found that media sector mentoring and training brought technical skills used to positive effect in that the debate format set a benchmark for competitor programmes. Spin-off programming included an hour-long call-in radio show on the debate points of Open Jirga.

The following is a part of the impact analysis of accountability: "In 2015, over a quarter (28%) of viewers and listeners strongly agreed that Open Jirga played a role in holding leaders to account. However, qualitative research revealed that satisfaction with panellists’ responses was mixed and notable cynicism persisted regarding the extent to which those in power were likely to respond to the issues raised. Some respondents felt unable to hold leaders to account outside the studio setting because of this, and due to the difficulty they faced in making contact with officials. Rural audiences in particular struggled to make a connection between the manner in which officials were questioned on Open Jirga and how they interacted with their local authorities. Some audiences and experts felt that Open Jirga could have done more to follow up to ensure that action was taken and demonstrate where power holders were failing in their duties."

Conclusions include: gains were severely hampered by the challenging social and political environment; knowledge of governance was increased among viewers; women's profile increased but access did not; individual-level outcomes of the debate and radio drama, "such as contacting local authorities, monitoring projects and resolving conflicts - were being replicated by listeners in their own communities"; technical skills were built but editorial skills training was not delivered; and a benchmark for debate programmes was set.