Evaluating Public Communication

Exploring New Models, Standards, and Best Practice

Jim Macnamara

• Publisher: Routledge
• Year: 2017
• Print ISBN: 9781138228573
• eBook ISBN: 9781315391960
• Topic: Social Sciences
• Subtopic: Media Studies
• Index: Social Sciences

Evaluating Public Communication addresses the widely reported lack of rigorous outcome and impact-oriented evaluation in advertising; public relations; corporate, government, political and organizational communication and specialist fields, such as health communication. This transdisciplinary analysis integrates research literature from each of these fields of practice, as well as interviews, content analysis and ethnography, to identify the latest models and approaches.

Chapters feature:

• a review of 30 frameworks and models that inform processes for evaluation in communication, including the latest recommendations of industry bodies, evaluation councils and research institutes in several countries;

• recommendations for standards based on contemporary social science research and industry initiatives, such as the IPR Task Force on Standards and the Coalition for Public Relations Research Standards;

• an assessment of metrics that can inform evaluation, including digital and social media metrics, 10 informal research methods and over 30 formal research methods for evaluating public communication;

• evaluation of public communication campaigns and projects in 12 contemporary case studies.

Evaluating Public Communication provides clear guidance on theory and practice for students, researchers and professionals in PR, advertising and all fields of communication.

Part 1
The foundations of evaluation

1
Why we need to Critically Examine Communication

It is tempting to jump right in and start talking about evaluation methods and the tingling satisfaction of proving that a public communication campaign or project was effective in achieving its objectives and generating results. This discussion will get into the nitty-gritty very soon – but, first, it is essential to understand and remind ourselves of a few very important things about communication.

Communication is recognized as fundamental to all aspects of human society. John Dewey famously said that ‘society exists . . . in communication’ (1916, p. 5). Raymond Williams echoed Dewey in saying that ‘society is a form of communication’ (1976 [1962], p. 10). However, communication between humans is a far more complex, uncertain and variable process than most of us care to admit. We know that communication is something that all humans do to some extent from soon after birth. But to assume that attempts at communication are always, or even mostly, effective is to ignore the wars, divorces, breakdowns of friendships, family fallings-out, feuds, misunderstandings and other disruptions that occur every day in human society.

In an address to aspiring communication students some years ago, I reminded them that:

Human history is one of lost letters, wrong numbers, downed wires, missed deliveries, misinterpreted messages, lost signals, no signal, failures to listen, incoherent speech and writing, mumbles, stammers and stutters, static, interference, not getting through, no one home, lost connections, errors, crashes, blackouts, breakdowns and dropouts, as much as it is of connection and communion.

(Macnamara, 2009)

Communication: the ‘registry of modern longings’

John Durham Peters poetically described communication as the ‘registry of modern longings’ (1999, p. 2). What he was referring to was the normative and aspirational expectations modern societies have of communication – what Pradip Thomas and Elske van de Fliert call the ‘ceaseless quest for a communication utopia’ (2014, p. 52). This involves an unfailing, but misguided, belief that communication will solve all of our problems and an unfounded confidence that communication will always work. Peters (1999) argues that we ask and expect too much of communication. We index almost everything against it to the extent that, when nations go to war or organization dissolves into chaos, we are surprised – as if human normalcy is a smooth flowing current of communication. Alas, that is not the case.

Peters says: ‘That we can never communicate like the angels is a tragic fact, but also a blessed one’ (1999, p. 2). He explains that giving up on the dream of instant unfailing communication is not at all to be driven into a nightmare of solitude or ignorance. He says that communication, in all of its fractures and mediations, is all we have. It is what makes us human. But the simple, inescapable and essential fact to recognize is: human communication often does not work.

It is surprising and somewhat frustrating that organizations, as well as individuals, continue to forget or ignore this. Our longing for, and faith in, the efficacy of human communication and a greatly inflated self-belief among many that they are ‘good communicators’ underlie the lack of evaluation in professional public communication practices that is the focus of this book.

Public communication

In contrast with interpersonal communication between two individuals (dyads) and within small groups, public communication refers to communication activities that take place in the public sphere (Habermas, 1989 [1962], 2006) rather than the private sphere (Chartier, 1989; Hansson, 2007) – albeit the separation of private and public is an increasingly blurred boundary in contemporary societies (Baxter, 2011). Also, public communication usually relates to matters of public interest rather than private affairs. Furthermore, those at whom it is aimed are commonly referred to as publics¹ (Eliasoph, 2004; Grunig & Hunt, 1984) and sometimes as target publics or target audiences. Individuals and groups with a vested interest in an organization, or which are directly affected by the activities of an organization, such as customers, shareholders, partners and affiliates, employees and sometimes local communities, are also referred to as stakeholders,² because they have a stake in the organization (Freeman, 1984). Sometimes, there are others who seek to have a stake or say in the activities of an organization and these are referred to in some literature as stakeseekers³ (Heath, 2002; Spicer, 2007). In the interests of simplicity, this text will refer to all such groups as audiences.

Recognizing that communication is meant to be two-way, it should be noted that organizations that seek to engage in communication with various publics and stakeholders are also audiences – or at least they should be, as I argued in Organizational Listening: The Missing Essential in Public Communication (Macnamara, 2016a). Communication requires speakers and listeners – or authors and audiences – to interact reciprocally.

Communication between organizations and individuals, such as letters, emails and complaints processed through customer service call centres, may not always be public in terms of being open for all to hear or see, but they are public in the sense of being part of organization–public relationships, abbreviated in public relations (PR) practice to OPR (Broom, Casey & Ritchey, 1997; Heath, 2013). Similarly, internal communication inside organizations, commonly referred to as organizational communication – although it would be more accurately called ‘intra-organizational communication’ – is also public communication. While management and employees engage in private interpersonal discussions in relation to individuals’ pay and performance, internal organizational communication addresses many or all members of an organization on matters of common concern. Internal organizational communication recognizes employees as key publics or stakeholders, and uses a range of public communication media and methods, such as the Internet, newsletters, videos, presentations, events and so on.

The website of the School of Communication at American University (2016) in Washington DC says of public communication: ‘It’s at the heart of our economy, society, and politics. Studios use it to promote their films. Politicians use it to get elected. Businesses use it to burnish their image. Advocates use it to promote social causes.’

Public communication campaigns are public communication activities that ‘use the media, messaging, and an organized set of communication activities to generate specific outcomes in a large number of individuals and in a specified period of time’ (Coffman, 2002, p. 2; Rogers & Storey, 1987, p. 821). Similarly, Charles Atkin and Ronald Rice say: ‘Public communication campaigns can be defined as purposive attempts to inform or influence behaviours in large audiences within a specific time period using an organized set of communication activities and featuring an array of mediated messages in multiple channels’ (2013, p. 3).

Despite their considerable cost in terms of both time and money, such campaigns are often not rigorously evaluated, partly because of the widespread assumption that communication efforts will have the desired effect This ignores the reality that others might have different views, beliefs, levels of literacy, and social and cultural influences. When hundreds of thousands, or even millions, of dollars, euros, pounds sterling or other currency units are spent on public communication, such as in major advertising, PR or public engagement campaigns, it is important to remember the famous statement by marketing pioneer John Wannamaker:⁴ ‘Half of my advertising is wasted; the only trouble is I don’t know which half’ (as cited by Albanese, 2007, p. 10, and in Stewart, Pavlou & Ward, 2002, p. 357).

Isabelle Albanese says that acceptance of Wannamaker’s dilemma ‘just doesn’t cut it anymore’ (2007, p. 10). A 50/50 chance of success is not acceptable in today’s age of accountability, tight budgets and performance management. Even if one accepts Wannamaker’s statement as a generalization, there is empirical evidence showing that public communication materials and campaigns, however creative and professionally packaged, regularly do not create the desired effects. A cover story in Advertising Age in 2006 reported market research that found 37.3 per cent of marketers’ advertising budgets to be wasted because they are ineffective (Albanese, 2007, p. 10). The following section will present further evidence that much public communication fails to achieve the intended outcomes. That is not an indictment of the ability or work ethic of the instigators of that communication. Rather, it is a reflection of the complex range of variables that affect public communication, the susceptibility of audiences to other competing and contrary influences and messages, and sometimes the resistance of audiences (Knowles & Linn, 2004) – factors that necessitate evaluation.

Evaluation can apply to specific campaigns, or even to specific activities within campaigns, such as media advertising, publicity, events, websites and publications. However, a single communication initiative such as an event or website, and even a campaign, rarely has significant outcomes and impact. In discussing health communication, D. Lawrence Kincaid and colleagues point out that a single campaign is usually only one phase in a series of campaigns and other types of intervention designed to achieve desired results (Kincaid, Delate, Storey, & Figueroa, 2013, p. 305). Learning from evaluation of campaigns should be incorporated into planning of future campaigns to gain incremental improvement. Thus it is productive to think of evaluation beyond the scope and time scale of any one campaign.

Furthermore, the concept of campaigns narrows the focus to planned purposive communication designed to achieve specific results desired by an organization. As Brenda Dervin and Lois Foreman-Wernet say, ‘no matter how carefully cloaked as attempts to “understand” and benevolently reach audiences, the intent of the campaign remains top-down social control – to entice audiences to comply with what experts deem appropriate’ (2013, p. 148). Public communication needs to be understood and considered more broadly as the ongoing communicative interaction between organizations and their various audiences – and vice versa. Today, public communication occurs 24/7, not only within the semi-controlled confines of campaigns. Public communication occurs every time an official gives a speech, every time someone posts a tweet on Twitter and every time a photo is posted on Facebook, Instagram or Pinterest. Public communication other than that planned by a particular organization occurs regularly, such as independent media reporting, public complaints, protests, petitions, social media comments and various other day-to-day expressions of the ‘will of the people’. Organizations need to monitor and evaluate what others say. Also, they often need to respond to communication initiated by others, such as letters, emails, website inquiries and social media discussion. Therefore, while this analysis of evaluation can be applied to campaigns, it discusses evaluation of public communication, not only of campaigns, recognizing that public communication is an ongoing, two-way process involving a range of channels and activities.

Throughout the twentieth century, it was common to refer to mass communication in discussing public communication with large groups of people. However, this understanding of communication is mostly associated with mass media, such as newspapers, radio and television, which dominated the mediascape during the twentieth century, as well as the questionable concept of mass society (Hoggart, 2004; Williams, 1976 [1962]). Public communication is a more inclusive term than mass communication because it applies to all channels used for direct and mediated communication, and it recognizes a range of audiences with differing interests and views rather than one ‘imagined’ mass audience (Anderson, 1991 [1983]).

Communication effects: direct, limited and contingent

Unfailing faith in the power of human communication, and particularly in the growing array of technologies such as broadcast radio and television that made so-called mass communication technically possible in the twentieth century, led to what is referred to as direct effects thinking and a belief in the strong effects of media and public communication. Such views have been referred to colloquially as the injection model, the hypodermic needle concept of communication and bullet theory (Schramm, 1971a; Seve...

Table of contents

• Cover
• Title
• Copyright
• CONTENTS
• List of figures
• List of tables
• Acknowledgements
• Introduction
• PART I The foundations of evaluation
• PART II The practice of evaluation
• PART III Case studies in evaluation
• References
• Index