Polling UnPacked

The History, Uses and Abuses of Political Opinion Polls

Mark Pack

• Publisher: Reaktion Books
• Year: 2022
• Print ISBN: 9781789145670
• eBook ISBN: 9781789145687
• Topic: Politics & International Relations
• Subtopic: Politics
• Index: Politics & International Relations

From a political-polling expert, an eye-opening—and hilarious—look at the origins of polls and how they have been used and abused ever since. Opinion polls dominate media coverage of politics, especially elections. But how do the polls work? How do we tell the good from the bad? And in light of recent polling disasters, can we trust them at all? Polling UnPacked gives us the full story, from the first rudimentary polls in the nineteenth century, through attempts by politicians to ban polling in the twentieth century, to the very latest techniques and controversies from the last few years. Equal parts enlightening and hilarious, the book requires no prior knowledge of polling or statistics to understand. But even hardened pollsters will find much to enjoy, from how polling has been used to help plan military invasions to why an exhausted interviewer was accidentally instrumental in inventing exit polls. Written by a former political pollster and the creator of Britain's foremost polling-intention database, Polling UnPacked reveals which opinion polls to trust, which to ignore, and which, frankly, to laugh at. It will change the way we see political coverage forever.

1

THE FIRST POLITICAL POLLS

‘A political leader in a general election, apart from being chronically overworked and unable to study even the daily press very carefully, is like a warrior or a pilot operating in almost total fog . . . [and so] market research has a valuable role to play.’

– HAROLD WILSON, prime minister¹

Political polls are part of the basic rhythm of democratic politics. But it was not always that way, for political polls are a much more recent invention than democracy. Democracy dates back over two millennia, while political polling dates back only two centuries to one particular election in the United States of America.

The United States

The U.S. presidential election of 1824 was odd. Only one party contested it. Yet a contest it was, with four different candidates from that one party competing for the public’s votes. Those votes, however, did not decide the election. Instead, with no candidate winning a majority in the electoral college, the decision went to the House of Representatives. The winner to emerge from there – John Quincy Adams – had led in neither the electoral college vote nor the popular vote.

Adams’s victory was a watershed in the development of the U.S. party system. The Democratic-Republican Party (not to be confused with either the later Democratic Party or the later Republican Party) had been dominant, and at the previous election in 1820, its candidate, James Monroe, did not even face a significant opponent.

The party had given the power to pick its candidates for president to its congressional delegation. With little in the way of rivalry from other parties threatening the success of the candidates it picked, that meant the presidential election had, in effect, become a small, closed choice by its congressional delegation alone.

That transfer and concentration of power in the hands of a small caucus was by 1824 widely mocked as ‘King Caucus’, in reference to the sort of monarchical power that the founders of the USA had deliberately tried to avoid. As a result, in 1824 only a quarter of the congressional delegation chose to take part in selecting the party’s official candidate for president. Moreover, three others from the party ran for president.

This shift of power away from elite decision-makers was enhanced by individual states increasingly moving to let the public vote in presidential elections. Previously, individual state legislatures had picked their state’s national electoral college members, and the electoral college then voted between candidates for president. But increasingly, states allowed public votes to determine the make-up of their electoral college members. In 1800 only 5 out of 16 states had such voting. By 1824 this had risen to 18 out of 24, and in 1836 South Carolina was the only holdout.²

These three factors – a proper, albeit confusing, contest with a real doubt over the outcome; the fracturing of the power of the Democratic-Republican Party’s congressional delegation; and the increasing use of the popular vote to decide the outcome in individual states – meant that there was more interest than usual in what the public (or, at least, the subset of men allowed to vote) thought about the candidates.

As a result, the contest also saw the precursors to modern political opinion polls. These were straw polls, simple tallies at various events, such as Fourth of July celebrations, of how many people supported each candidate. In some cases, books were left out in a public place for several days, with people writing in their voting choice. The primary locations for straw polling, however, were local militia meetings. Militia musters were ideal occasions for such votes since the enrolled militia covered all white males aged 18 to 45 and the annual musters were popular events that attracted large crowds. Asking about elections at them fitted with the militias’ democratic and political traditions. Most state militias elected their lower offices and some even elected higher ones. They were a natural and relatively straightforward way of testing public opinion. In a foreshadowing of later polling controversies, however, there were arguments over how typical those attending militia meetings were of the broader public, given that one candidate, Andrew Jackson, was a hero of the militia.³

There were none of the techniques described elsewhere in this book to ensure a representative sample. Although some reporting tallied up scores from different events to help give a picture of which candidate could claim to be the ‘public’s’ choice, these were much cruder affairs than modern polling.⁴

The exact details of these straw polls are often misreported, sometimes described as organized by newspapers and sometimes giving the credit to newspapers that were actually just re-reporting what other papers had already covered. But it appears that three newspapers – the Raleigh Star and North Carolina State Gazette, the Wilmington American Watchman and Delaware Advertiser and the Star and North Carolina Gazette – deserve the credit for the most pioneering reporting of the straw polls.⁵

In another foreshadowing of later disputes about political polling, there was a pattern of different newspapers liking or not liking straw polls depending on how the results did or did not line up with their own editorial line. The Raleigh Register, for example, was a supporter of candidate William H. Crawford and attacked the act of ‘prematurely collecting the opinion of people’. Those collections of opinions were showing Crawford lagging.⁶ Political polling, in its earliest form, was up and running.⁷

The next significant development again came in the United States, this time in the late nineteenth century with the Columbus Dispatch. In a step up from the straw polling that had continued since 1824, the paper trained its interviewers, sending them out across the city. It even considered the age and occupation of those interviewed to (try to) make the results representative. But these pioneering efforts did not catch on, and although straw polling continued, with increasing numbers of votes being counted up, the Dispatch’s methodological sophistication remained the exception. Moreover, such results remained geographically constrained. There were no attempts to gauge support across the whole country.

This changed in the early twentieth century with the Literary Digest, a national magazine. It can be credited with both the introduction of the idea of seeking national results by asking people across the whole country and then, years later, the embarrassment of a failure so significant that it helped establish modern polling.

The Literary Digest started posting out questions to people in selected areas in 1916, later spreading to the whole country.⁸ As with the Columbus Dispatch, the Literary Digest showed some understanding of the need to make sure that the people asked were representative of the wider public. But it mainly selected people from limited lists, such as those with telephones or club memberships, or those who had registered cars.

What appeared to give the magazine’s surveys authority, and what made them famous, was the exceptionally large number of responses involved. Questions about Prohibition in 1930 got answers from 5 million people. This impression of trustworthiness was reinforced by the Literary Digest’s record at elections. At five presidential elections in a row (1916, 1920, 1924, 1928 and 1932), its survey correctly called the winner.

The Literary Digest therefore became the most famous straw pollster. But it was by no means the only one. Nor was the straw polling carried out by dullards who did not understand the risks of biased results. Far from it. Comparisons between straw poll findings and election results were common and there were many variations in methodology as straw pollsters sought to get the best results. This even included what we would now recognize as methodological rigour, such as sometimes posting out a ballot to every tenth person on the electoral roll. Those undertaking such surveys may not have understood the mathematics of sampling, but they were groping towards doing the right thing, just as medicine has often seen treatments evolve before a proper understanding has been acquired of what makes them effective.

For careful observers, however, there were signs of frailty. As one pre-1936 critic pointed out, only one of the five elections for which the Literary Digest had accurately predicted the winner – the election of 1916 – had been a close contest. It is much easier to get five in a row right when four of them are landslides. Moreover, the Literary Digest’s record on predicting vote shares, rather than just the winner, was less impressive.⁹ Despite those signs of frailty, the involvement of millions in its surveys and its record of picking the winners meant that by 1936 the magazine claimed that ‘the Digest poll is still the Bible of millions’.¹⁰

Then came the 1936 U.S. presidential election. Ten million people across the country were mailed, and 2,266,566 returns were received. They showed a landslide for Republican Alf Landon, with a 57–43 per cent lead over Democrat and incumbent president Franklin D. Roosevelt. As the magazine reported,

For nearly a quarter century, we have been taking Polls of the voters in the forty-eight States, and especially in presidential years, and we have always merely mailed the ballots, counted and recorded those returned and let the people of the Nation draw their conclusions as to our accuracy. So far, we have been right in every Poll . . . The Poll represents the most extensive straw ballot in the field – the most experienced in view of its twenty-five years of perfecting – the most unbiased in view of its prestige – a Poll that has always previously been correct.¹¹

The Literary Digest further boasted that its numbers were unadulterated. It made a virtue of this in a way redolent of how food manufacturers now boast of products being natural and additive-free: ‘these figures are exactly as received from more than one in every five voters polled in our country – they are neither weighted, adjusted, nor interpreted.’

But this was the pride before the fall.¹² For the actual election result was a crushing landslide – for Franklin D. Roosevelt. Rather than losing by 43–57 per cent, he was easily re-elected by 61–37 per cent. Both his vote share and his electoral college result were the best for any candidate since 1820.

As, to its credit, the magazine’s later headline put it, ‘Is our face red!’ – although a bombastic determination to continue with such surveys followed: ‘Should the University of Minnesota, with the greatest record in modern football, give up the sport because it finally lost one game, after a string of twenty-one victories?’¹³

What had gone wrong? Part of the answer is that the people asked to take part were more Republican than voters overall are. Sources the magazine used, such as car owners, skewed towards those with more money. This particularly mattered in 1936 because the election was held during a severe economic downturn and contested between candidates with very different platforms for helping the poor. However, this explanation on its own is not sufficient as, for example, in Chicago the Literary Digest mirrored modern sampling by asking every third registered voter. Yet its forecast of Landon’s support was too high there as well. The other part of the answer was an issue with how willing each candidate’s supporters were to participate in such surveys. This early form of the ‘non-response bias’ contributed roughly twice as much to the overall error as did the problem of asking people who were more Republican than voters overall.¹⁴

Yet while the Literary Digest was spectacularly wrong, a new entrant was spectacularly right, using the results from a smaller number of people – a much smaller number: around 50,000 compared with 2.3 million. Those results were, however, not left unweighted, unadjusted and uninterpreted as if raw data was a virtue. They were instead weighted, adjusted and interpreted to create the truth.

The new entrant was George Gallup, armed with a novel scientific method based on understanding the statistical theory behind sampling. His critical insight was that the number of respondents is not what makes a poll accurate. Rather, it is how representative those respondents are of the population you are looking to measure. We will look at the science – and art – of sampling in more detail later. For the moment, it is enough to know that if, say, you want to discover what football clubs people support, standing outside Newcastle United’s St James’ Park to ask your question will ...

Table of contents

• Front Cover
• Half Title
• Title Page
• Copyright
• CONTENTS
• Introduction
• 1. The First Political Polls
• 2. How Political Polls Work
• 3. The Rights and Wrongs of Polling Questions
• 4. Is It All about Voting Intention?
• 5. Other Types of Polls
• 6. Polling Gets It Right
• 7. Polling Gets It Wrong
• 8. Regulating Opinion Polls
• 9. Alternatives to Opinion Polls
• 10. Even Good Polls Mislead
• 11. Judging Polling Firms
• 12. Making Sense of the Polls
• Thank You
• References
• Further Reading
• Acknowledgements
• Index