News, Numbers and Public Opinion in a Data-Driven World

An Nguyen

• Publisher: Bloomsbury Academic
• Year: 2017
• Print ISBN: 9781501354007
• eBook ISBN: 9781501330360
• Edition: 1
• Topic: Social Sciences
• Subtopic: Communication Studies
• Index: Social Sciences

From the quality of the air we breathe to the national leaders we choose, data and statistics are a pervasive feature of daily life and daily news. But how do news, numbers and public opinion interact with each other – and with what impacts on society at large?

Featuring an international roster of established and emerging scholars, this book is the first comprehensive collection of research into the little understood processes underpinning the uses/misuses of statistical information in journalism and their socio-psychological and political effects. Moving beyond the hype around "data journalism," News, Numbers and Public Opinion delves into a range of more latent, fundamental questions such as:

· Is it true that most citizens and journalists do not have the necessary skills and resources to critically process and assess numbers?

· How do/should journalists make sense of the increasingly data-driven world?

· What strategies, formats and frames do journalists use to gather and represent different types of statistical data in their stories?

· What are the socio-psychological and political effects of such data gathering and representation routines, formats and frames on the way people acquire knowledge and form attitudes?

· What skills and resources do journalists and publics need to deal effectively with the influx of numbers into in daily work and life – and how can newsrooms and journalism schools meet that need?

The book is a must-read for not only journalists, journalism and media scholars, statisticians and data scientists but also anybody interested in the interplay between journalism, statistics and society.

Section One

Data and Statistics in News Production

1

Common Statistical Errors in the News: The Often-Unfulfilled Roles of Journalists in Statistics–Society Relationship

Fabienne Crettaz von Roten , University of Lausanne, Switzerland

Introduction

Journalists hold a crucial function in the proliferation of numerical and statistical elements in the public sphere, as they introduce such information in daily news reports of virtually all areas of society. For example, audiences are frequently exposed to probabilistic estimates of a new disease, confidence intervals of success of a politician for an upcoming election, or results of a survey on a social issue. As one statistician observes, ‘I find it hard to think of policy questions, at least in domestic policy, that have no statistical component’ (Moore 1998, 1253). Understanding these statistical components is crucial for citizens to participate in public debate and to make political decisions. H. G. Wells’s prophecy of a century ago – that statistical thinking will one day be as necessary for citizenship as the ability to read and to write – is becoming a reality (Billard 1998). But, as I argued elsewhere, ‘since most citizens lack statistical literacy, it is not surprising that ambivalence is increasing and that the better educated (with more statistical skills) are able to develop more firmly held, more extreme attitudes’ (Crettaz von Roten 2006, 247). For modern data-rich society to operate effectively, citizens must be statistically literate to avoid exclusion.

As the news media are a major source of numerical and statistical elements, journalists hold a crucial function in a data-rich society, through their daily work of selecting numerical and statistical information and translating this information into news stories that contribute to the formation of attitudes. As the long tradition of social psychology demonstrates, people form attitudes towards an object by way of information at disposal, the social environment, salience issues and so on (Eagly and Chaiken 1993). As such, the quality of statistical information in the media could exercise a strong influence on the relationship between statistics and society, and more generally between science and society. Mathematical and statistical errors or misunderstandings in the news media, however, are all too common.

As an engaged statistician, that is, a scientist willing to use her expertise for the public good,¹ I often feel the duty to write to editors or journalists when I see a statistical problem in the news. My contact with journalists frequently leaves me surprised and frustrated: journalists either stick to their positions, or reject the related errors with arguments that do not refer to statistics but other aspects of news. Many would argue, for example, that aesthetic considerations can take on the correct processing of information, that it’s important to increase the impact of the news even with incorrect charts or that the reader would not understand otherwise. This chapter reflects on that experience to highlight three roles that, in my views, journalists should endorse in dealing with the relationship between statistics and society, namely verifying statistical information, presenting statistics in an accurate manner in the news and providing statistical education for the public. I will start with a personal account on common misuses of statistics in the media with examples from my own country – Switzerland – before moving on to briefly discuss each of the three roles. If these roles call for a rigorous inclusion of statistical thinking in journalism training, this has hardly translated into a reality in journalism education. The final section, therefore, reviews the place of statistical training in journalism education and suggests some ideas for future journalists to shoulder the three roles and to improve the relationship between statistics and society.

Some common misuses of statistics in the media

Statistical literacy can be characterized as ‘the ability to understand and critically evaluate statistical results that permeate our daily lives – coupled with the ability to appreciate the contributions that statistical thinking can make in public and private, professional and personal decisions’ (Wallman 1993, 1). According to Marriott (2014), the core concept of statistical thinking has evolved in the past century: at the beginning of the twentieth century, it involved averages, minimum and maximum; in the late twentieth century it expanded to expectation, variance, distribution, probability, risk and correlation; and in the twenty-first century it adds data, visualization and cognition. In this context, statistical literacy among journalists should ideally have followed the evolution. But scholars and critics have long documented poor and wrong uses of statistics in the media (Huff 1954; Paulos 1995; Best 2001,Best 2005; Goldin 2009; Nguyen and Lugo-Ocando 2016). Here are a few common misuses that I find from my own experience:

1 Much news involves figures found in releases or documents, but some numbers are so erroneous that they indicate a lack of order of magnitude among journalists. For example, one newspaper reported that 77.8 per cent of employees – instead of 7.5 per cent – had been harassed according to a study of the State Secretariat for Economic Affairs (Le Matin 2011); another estimated the remaining costs of tunnelling for the Lötschberg tunnel at 1.3 million Swiss francs instead of at 1.3 billion Swiss francs (Le Matin Dimanche 2010); finally, an article mentioned that Africa’s gross domestic product was $2 billion instead of $2,000 billion (Le Temps 2012a).

2 Journalists often fill the news with figures from statistics providers, such as EUROSTAT, but such data series might not be fully comparable (e.g. not the same year, not the same statistics), which is often explained in providers’ metadata files. To use the data, caution should be, but is rarely, exercised to read these files fully and critically. For instance, a Swiss newspaper compared gross annual salaries among European countries from EUROSTAT, without noting that gross salaries were summarized by a mean in some countries and by a median in others,² causing misleading comparisons and a misleading conclusion. In this example, the problem is not the quality of the data (the information is mentioned by the statistics provider) but lies in the journalist’s use of the data. In my experience, journalists fail very often to deliver such notifications or mention them in very small print, making it difficult for readers to realize the difference of the measures at stake.

3 When journalists report the results of surveys, such as of a pre-electoral survey, they usually provide the accurate percentages of intended votes for candidates, but some imperfectly refer to information about variability. They endlessly comment on a 2 per cent or 3 per cent difference betwee n two candidates, or on a 2 per cent or 3 per cent increase of voting intention for one candidate. This is meaningless if the margin of error is greater than 3 per cent – and this piece of technical information is often either unreported or shown in small print at the end of the article.³ Even worse case is when they compare groups, such as linguistic regions in Switzerland, whose sample sizes would be smaller and the margin of error is bigger (often around 6 per cent). This misinterpretation of statistics is more or less frequent, varying across the types of media and the countries, but in Switzerland it is an ongoing problem.

4 Some technical terms in statistics, such as ‘normal’, ‘significant’, ‘random’ or ‘correlation’, are frequently used for a non-technical meaning in daily life, creating a common source of confusion. Its consequences can be seen through a recent news story about the price increase of roasted chestnuts (Tribune de Genève 2012). The journalist begins with the presence of a parasite in chestnuts (dryocosmus kuriphilus), then talks about droughts in Italy – the main supplier of chestnuts for Switzerland – and continues with the following comment: ‘This meteorological phenomenon was correlated with the arrival of a devastating parasite.’ What does ‘correlate’ stand for here? A non-technical term used only as a synonym of ‘coincide’, or a technical term, that is, a point-biserial correlation or a causal relationship (the more the weather is dry, the more we observe parasites in chestnuts)?⁴ Such terms, in my views, should be used with caution (as little as possible), with an indication of its sense (technical or non-technical, and, for the former, the specificities of the technical sense).

5 The concepts of p-value and statistical significance play a central role in reporting quantitative results. For a statistician, these notions have a precise definition in the theory of statistical tests and it is recommended to complete it with effect sizes, whereas for a journalist, they are often only a rhetorical argument. An impressive number of scientific articles list the misuses of the notions and provide their exact meaning – see, for example, Glaser (1999) and Greenland et al. (2016). The media report many significant results, often wrongly as a synonym of practical importance or of causality, even if the design of the research does not allow that. The situation becomes more complicated if one considers that ‘the adjective “significant” has a different meaning in the mouth of a scientist and on the ears of the general public’ (Duplai 2012, 15). An article about household composition in Geneva stated that ‘the structure of the household has experienced two significant changes between 2000 and 2011: the proportion of couples without children decreased by 23% to 21% and lone parent households increased by 7% to 9%7 per cent to 9 per cent’ (Tribune de Genève 2015). What is the meaning of ‘significant’ here? If it is related to a statistical test, the reader needs more information on the context to be able to judge the practical importance of the changes (how many couples without children, how many one-parent families, what effect sizes and so on).

6 Graphs are often used to illustrate a news text with numerical and statistical elements. Data visualization has become more and more professionalized, thanks in a large part to technological advance and to leading figures in academic spheres (e.g. Hans Rosling, professor of international health at Karolinska Institute, Sweden) and media spheres (e.g. Amanda Cox, New York Times graphics editor). But misleading visuals can still be found, as was a case in a magazine for a Swiss university community (Wyss and Meyer 2013). The article addressed the distribution of men and women in academic staff of the university and included male/female silhouettes instead of bars (i.e. a bar chart) to illustrate the gender difference. The height of the silhouette was proportional to the data but the width of the silhouette was proportional to the height, leading to a surface that is not proportional to the data. This choice resulted in accentuation of the deficit of women in higher status. When contacted, the journalist acknowledged the error but justified it by citing the need of identification for the reader (more identification to a silhouette than a bar) and for aesthetic purposes. He concluded that the purpose was to raise awareness of the underrepresentation of women in leading positions and his graph achieves this perfectly.

This list is not at all exhaustive, but it is substantive enough for us to be concerned about the state of statistics in the media. This is worrying in light of recent changes to journalism in Western societies. Declining numbers of journalists and increasing time constraints on news production have increased the pressure on journalists to do more in less time, with a shortening of the verification step, meaning more factual errors,...

Table of contents

• Cover
• Half-Title
• Title
• Contents
• List of Contributors
• Foreword Stuart Allan
• Introduction: Exciting Times in the Shadow of the ‘Post-Truth’ Era: News, Numbers and Public Opinion in a Data-Driven World
• Section One Data and Statistics in News Production
• Section Two Data and Statistics in News Consumption
• Section Three Agenda for the Future
• Index
• Copyright