How to Question Numbers

"[W]hen journalists report numbers well, their stories gain depth, accuracy and influence. Overcoming your fear and distrust of numbers will make you a better journalist."

Designed for science journalists, this SciDev.net guide explains how to get to grips with numbers, focus on budgets, and assess numerical goals announced for new government projects. It was written by journalist Linda Nordling, who specialises in African science policy, education, and development. It is based on her observation that, "even when reading a data-rich scientific paper, most of us rely more than we care to admit on scientists' conclusions, rather than cross-checking with the data....[T]hat mindset serves journalists poorly when writing about government budgets and initiatives to build scientific capacity in the country where they live, which is an increasingly important side of being a science journalist." In this context, Nordling suggests:

• Watch out for "single numbers", such as a single headline number like "we will spend one per cent of the Gross Domestic Product (GDP) on science and technology (S&T) by 2015." Nordling argues that such single figures, especially when they are sold as ambitious targets set by governments, need other figures to provide context. "One per cent might not be an increase at all....Ask press officers or officials for comparative data. Accepting figures uncritically is very poor journalism."
• Do your arithmetic: "Let's say the Bill & Melinda Gates Foundation announces US$20 million for hiccups research, and you report figures allocated to each country. If your breakdown adds up to US$18 million, readers will query the remaining US$2 million (and if the parts sum to over US$20 million, you will look foolish)."
• Put numbers in context: "For instance, a 20-acre wind farm may be difficult to picture. Easier is to say that the farm will be the same size as 'about ten football pitches'."
• Question numbers: "A few years ago, the government of Ethiopia announced a plan to produce 5,000 new PhDs over a decade. A little bit of research showed the country had produced fewer than 100 PhDs in the previous 50 years....In fact, the programme pivoted around a plan to invite foreign academics to teach and supervise the PhDs....[C]hecking figures gave the reader more information about Ethiopian science and put the government's ambitions in context."
• Check others' work: "Trust the 'this doesn't make sense' feeling at the pit of your stomach and start asking questions. Perhaps the press release is wrong. Perhaps there are plans for raising the rest of the money that you've not been told about. Step back and ask 'do things add up?'."
• Diligently and thoughtfully engage with numbers, which "can identify new story angles, and even get you a scoop."
• Use editors and subeditors to help probe your conclusions or talk numbers through with a colleague. "If you mess up your calculations and run with a story based on bad maths then you might find government officials or donor press officers less than helpful in the future."