Measurable Journalism

Digital Platforms, News Metrics and the Quantified Audience

Matt Carlson, Matt Carlson

• Éditeur: Routledge
• Année: 2020
• ISBN:
  9781000706772
• Édition: 1
• Sujet: Médias et arts de la scène
• Sous-sujet: Biographies de journalistes

This book explores ways in which the increasingly 'measurable' news audience has had an impact on journalistic practices, in an era when digital platforms provide real-time, individualizable, quantitative data about audience consumption practices.

Considering the combination of digital technology that makes measurable journalism possible, the contributors to this volume examine the work of various actors involved in aspects of measurable journalism both inside and outside the newsroom and confront the normative implications of the data-centric trends of measurable journalism. Including examples from across the globe, the book balances hopes for increased engagement or impact with fears that economic prioritization will hurt journalism's standing in the public sphere.

This book will be of interest to those studying journalistic practices in the modern world, as well as those studying media consumption and emerging digital technologies. This book was originally published as a special issue of Digital Journalism.

QUANTIFIED AUDIENCES IN NEWS PRODUCTION

A synthesis and research agenda

Rodrigo Zamith

A number of social, technological, and economic shifts over the past two decades have led to the proliferation of audience analytics and metrics in journalism. This article contends that we are witnessing a third wave toward the rationalization of audience understanding and distinguishes between audience analytics (systems that capture information) and audience metrics (quantified measures output by those systems). The body of literature on analytics and metrics in the context of news production is then synthesized across the ABCDE of news production: attitudes, behaviors, content, discourse, and ethics. That synthesis leads to an overarching conclusion that while contemporary journalism is not being driven by quantified audiences, both audiences and quantification are playing far more prominent roles in news production than in the past. Scholars and practitioners have also become less pessimistic about analytics and metrics over time, recognizing more nuanced effects and prosocial possibilities. Finally, important gaps in the literature are identified and new research directions proposed to help address them.

There has been a movement in media industries over the past 90 years toward ever-greater rationalization of audience understanding, or the use of scientific methods to construct audiences based on data (Napoli 2011). This movement has manifested itself most recently in the proliferation of audience analytics, systems that capture a range of audience behaviors, and audience metrics, quantified measures from which preferences are inferred. Scholars have taken great interest in these developments within the context of journalism, with some arguing that it may lead to “a fundamental transformation … in journalists’ understanding of their audiences” and perhaps ultimately toward a journalism driven by the “agenda of the audience” (Anderson 2011b, 529). That shift may also lead to greater emphasis on personalized news experiences that focus on individuals rather than communities (Anderson 2011a), posing challenges to the development of common knowledge and publics (Tandoc and Thomas 2015); changes to the authoritative and jurisdictional claims journalists are able to make (Lewis and Westlund 2015b); and to the reworking of boundaries that are fundamental to the self-understanding of professional journalism (Coddington 2015). However, scholars have long observed that the availability of a technology does not mandate its use (Pinch and Bijker 1984). Affordances must be analyzed in their social, historical, and economic contexts in order to understand the diffusion, acceptance, and use of a technology, which may then be used in myriad ways (Siles and Boczkowski 2012). While the potential for transformation is considerable, scholars are still disentangling the impacts audience analytics and metrics are having on contemporary news production.

The aim of this article is to situate the increasing quantification of audiences within broader theoretical and historical contexts, synthesize the scholarship on audience analytics and audience metrics, and highlight areas for further development in that stream of work. It is argued that we are witnessing a third wave toward the rationalization of audience understanding that is both distinct and in some ways a continuation of pushes in the 1930s and 1970s to use scientific methods and technological innovations to better quantify audience preferences and behaviors. Audience analytics are ubiquitous in today’s newsrooms, with many utilizing multiple systems. While contemporary journalism does not appear to be driven by audience metrics, they are now factored to some extent into journalistic attitudes, behaviors, content, discourses, and ethics. Following an initial period of skepticism and pessimism, there is now growing optimism about and acceptance of metrics among both practitioners and scholars. However, a number of critical questions remain unanswered within this stream of work.

The article begins by explaining the notion of constructed audiences, historicizing the construction of audiences by media companies, describing the potential that audience analytics and metrics offer for transforming those constructions, and situating the rapid proliferation of those systems and measures within social and economic developments. The budding scholarship on this phenomenon is then distilled to outline the impacts of quantified audiences on the ABCDE of news production: attitudes, behaviors, content, discourses, and ethics. Finally, that stream of work is evaluated and critical questions that remain unanswered are highlighted.

Toward Quantified Audience Constructions

A long line of scholarly work has examined audiences as socially constructed entities. A constructed audience refers to the “images” (Gans 1979) and “abstractions” (Schlesinger 1978) developed by media producers of the individuals that make up an audience. These interpretations emerge in the minds of newsworkers through exposure to different inputs over the course of day-to-day activity. It is important to distinguish between constructed and actual audiences because the former may reflect the latter poorly, such as in terms of size, make-up, interests, and information needs. Indeed, newsworkers long depended on letters to the editor and interactions with their immediate peers and friends as primary inputs for their construction of the audience, yielding abstractions that were only marginally reflective of those who consumed their work (DeWerth-Pallmeyer 1997; Gans 1979).

Crucially, a social constructivist perspective and social psychological theories like the Theory of Planned Behavior contend that individuals can only make decisions based on their perceptions of phenomena. For journalists, many of those perceptions stem from their tacit professional knowledge, which they do not actively think about during their work and have trouble easily articulating (DeWerth-Pallmeyer 1997). Constructed audiences, in particular, inform decision-making at multiple levels, from calculations of newsworthiness (Wallace 2017) and noteworthiness (Napoli 2011) to organizational strategy (Turow 2005).

Constructed audiences are therefore capable of influencing both conscious and subconscious decisions. It is thus important to ascertain how such images come to be and how the process of abstraction has changed over the past several decades to emphasize quantification.

Modern Rationalization of Audience Understanding

According to Napoli (2011), there are two interrelated processes that drive changes to the rationalization of audience understanding, or the use and refinement of empirical, typically quantitative techniques to aid the understanding of multiple dimensions of audience behavior in order to better predict and respond to those behaviors. The first involves technological changes that alter the dynamics of media consumption. The second involves technological changes that facilitate the gathering of new forms of information about the media audience. To these, one should add social and economic changes that alter conceptualizations of and discourses around audiences as well as the imperatives for serving and monetizing them (Anderson 2011b; Turow 2005).

Napoli (2011) observes that media industries’ perceptions of their audience became increasingly scientific and data-driven over the course of the twentieth century. Two waves of audience measurement developed during that time. The first began in the 1930s when media organizations started moving away from a then-dominant “intuitive model” whereby decisions were made based on “subjective, often instinctive, judgments … regarding audience tastes, preferences, and reactions” (p. 32). Economic hardships drove advertisers to demand “tangible” evidence of effectiveness and news organizations began collecting data on their readers’ demographic and behavioral characteristics using scientific methods like systematic reader surveys. A second wave emerged in the 1970s as computers facilitated the collection and analysis of larger quantities of statistical data, news consultants were brought in to help attract larger audiences, and managers sought additional quantitative data to help them make more “scientific” managerial decisions (Napoli 2011).

This history underscores the fact that neither audience measurement nor the growing inclination to incorporate audience feedback into editorial decision-making are novel phenomena. As Nadler (2016) argues, the popular perception that digital journalism “represents a historical rupture” (p. 2) vis-à-vis the desire to let users’ preferences set the agenda of news organizations is misguided. The first wave occurred during journalism’s “high modernism” period, when it was dominated by a culture of professionalism and driven by the conviction that journalism’s primary function was to serve society by focusing on “objective” information about public affairs (Hallin 1992). As such, important cultural and institutional barriers that emphasized professional autonomy restricted the impact of the newly collected audience information on editorial activities. In contrast, the second wave during the 1970s was part of (and helped drive) a paradigmatic shift in the field away from the ideal of professional autonomy and into a “postprofessional” period marked by greater institutional acceptance of the idea that consumers’ preferences should factor into news production (Nadler 2016). Unlike its predecessor, the second wave took place during a period of perceived economic insecurity among news companies—though they largely remained high-profit enterprises—and mounting pressures for increased revenues, which led to managerial pushes toward more market-driven journalism (DeWerth-Pallmeyer 1997). There was, therefore, greater incorporation of systematically collected audience feedback into editorial decision-making than in the past, though frontline newsworkers (e.g. lower-level editors and journalists) at many organizations continued to largely reject it (Gans 1979).

While the “postprofessional” period has indeed extended to the digital age (Nadler 2016), the media environment began to see notable technological, economic, and social transformations in the early 2000s that have culminated in a third wave toward the rationalization of audience understanding. This wave is characterized by the development and rapid proliferation of low-cost, automated systems that can capture, link, and organize large amounts digital trace data that reflect non-purposive feedback from all consumers of digital media products (Mullarkey 2004). It is also characterized by new discourses around the term “audience,” namely in terms of its role within news production and how it is articulated through a range of quantifiable measures (Anderson 2011a). This wave is partly a product of economic upheaval in the industry (Lowrey and Gade 2011) as well as the changing nature of professionalism within the field (Meyers and Davidson 2016) and the “big data” phenomenon that extends beyond it (Lewis and Westlund 2015b). Finally, this wave has introduced real-time audience feedback to a far larger range of actors and activities within news production than its predecessors.

With regard to the first of Napoli’s (2011) two interrelated processes, it is important to note that the contemporary media environment is distinguished by fragmentation and audience autonomy. There is presently a large and growing array of content delivery platforms, resulting in the disaggregation of content and the diffusion of audience attention (Napoli 2011). Audiences now have considerably more control over how they consume media and can produce their own content at marginal costs, giving them greater autonomy and more choices (Bruns 2008). These shifts have created significant challenges for traditional audience information systems—the “data gathering and feedback mechanisms used … to measure audience exposure to media content … predict content preferences … target content … and gather information on audiences’ reactions” (Napoli 2011, 10)—since they struggle to capture such dispersed and empowered audiences.

Audience Analytics and Metrics

Central to this third wave are audience analytics. Though the term is sometimes used interchangeably with audience metrics in the scholarly literature, there is value in distinguishing between them. Audience analytics refer to the systems and software that enable the measurement, collection, analysis, and reporting of digital data pertaining to how content is consumed and interacted with (see also Braun 2014). ¹ They include the algorithms that log data requests and capture a range of user actions (e.g. how far they scrolled down a page), aggregate data to highlight patterns or make recommendations (e.g. trending stories), and present information about an audience via an intuitive interface (e.g. online dashboard). There are several such systems—Chartbeat, Google Analytics, and Parse.ly are among the most common today—and they are often used in conjunction with one another (Cherubini and Nielsen 2016). They are sometimes supplemented with a custom-built system specific to the news organization or developed by a parent company. Unlike previous audience information systems, audience analytics do away with the need to sample and capture information that may be omitted from self-reports, making it “possible to record data about individual consumers at an unprecedented level of detail” (Mullarkey 2004, 42).

Audience metrics refer to the quantified and aggregated measures of audience preferences and behaviors generated by those data collection and processing systems (see also Zamith 2016). Nguyen (2013) notes that there are two distinct sets of metrics: internal and external. Internal metrics include data about how a site or app is utilized by users during their visit, including data about traffic to and from the organization (e.g. number of unique visitors) and about user behaviors (e.g. number of times the share button is clicked). External metrics consist of information about preferences and behaviors occurring on other platforms (e.g. trending keywords on Twitter).

The distinction between systems (analytics) and output (metrics) helps separate the artifactual nature of a technology and the textual nature of its content (see Siles and Boczkowski 2012). A single analytics system may output multiple metrics. The same metric may be captured and analyzed by multiple systems (sometimes under different labels). The meanings associated with a metric can change even as the system that produced it remains stable, and vice versa. Furthermore, metrics can come to carry meanings that are very different from what the creators of the analytics that enable them imagined (see Orlikowski 2000; Pinch and Bijker 1984).

While different systems may focus on some of the same metrics, they employ different algorithms to collect, synthesize, and present that information. Two systems may—and often do (Cherubini and Nielsen 2016)—provide different information about the same phenomenon (e.g. trending stories). For example, an “engagement” metric may be operationalized differently across systems even as they use the same label. Different systems may thus generate and present very different abstractions of audiences to newsworkers, which in turn shape distinct constructions in the newsworkers’ minds. The disconnect can confuse newsworkers (Graves and Kelly 2010) and lead them to source particular measures from particular systems with limited regard for the consequences.

In combination, audience analytics and audience metrics offer the potential to dramatically alter editorial newsworkers’ constructions of audiences by introducing powerful new inputs. They offer a real-time look at an array of information about individual actions and population-wide (in a sampling sense) behavioral patterns that sometimes challenges the “gut feelings” journalists draw upon (DeWerth-Pallmeyer 1997; Hanusch and Tandoc 2017). The mythology surrounding “unbiased” data and the “science” of algorithms has also led many practitioners to believe that audience analytics can narrow gaps between constructed and actual audiences (Cherubini and Nielsen 2016; MacGregor 2007). However, it is important to note that audience analytics generally capture select behavioral data, from which beliefs and attitudes can only be inferred. While those systems offer more granular data for certain behavioral phenomena, they may (and often do) offer less data about what audiences are thinking than prior...