1
SOLVING THE PUZZLE AN INTRODUCTION
Politicians, most people think, will say anything to get elected. Parties, if they are in opposition, will do whatever it takes to get into power and, if they are in government, whatever it takes to stay there. This book qualifies that common wisdom and tries to explain why it is often wrong. It does this by focusing on a party that for decades had a reputation for its ruthless pursuit of power but which, until very recently, seemed to have forgotten how to win. If at the heart of every story there has to be a question, then it is this: how and why did the Conservative Party, of all parties, not do what it had to do in order, firstly, to stay in Downing Street after 1992 and, secondly, to get back there as soon as possible after 1997? Just as importantly: why and how was it able, after 2005, to put things right so quickly? And why, despite that, has its recovery been partial rather than comprehensive?
This book’s task, then, is not so much to explain why the Conservative Party lost elections between 1997 and 2005 – there are, after all, any number of reasons why that can happen, many beyond the control of politicians, especially when they are in opposition and not government. Rather it tries to explain why those politicians were unwilling and unable to act in a way that might have given them more hope of winning or at least losing less badly during that time – and of winning much bigger than they actually did in 2010 and in 2015. In so doing, it argues that, while there is no point trying to find and then flog a simple, superficially attractive answer to such a difficult problem, we can nevertheless provide an explanation that is both realistic and intelligible – and one that not only works in this case but might get us thinking about others, too.
The key to such an explanation lies in realizing, firstly, that party politics, indeed all politics, is essentially the interaction of ideas, interests, institutions, and, of course, individuals – people who, however intelligent and well intentioned, are, like the rest of us, hardwired to make what, objectively speaking, are sometimes irrational decisions; people who are both the product and the producers of the organizations they work in and the ideas they work with. Secondly, we need to bear in mind that politics is ‘path-dependent’: things said or done early on can constitute ‘critical junctures’ which then make certain courses of action (even if they are misguided) almost inevitable and others (even where they would seem to make more sense) almost impossible.
Thirdly, it is worth paying attention both to the people who actually do politics (which is why this book makes use of extensive interviews with politicians at all levels of the Conservative Party) and to the people who make their living reporting on it. This is not simply because they can give us a crucial insight into what, significantly, is a very small world. It is also because the line between players and recorders in politics is a very blurred one. Indeed, one of the main contentions of this book is that one cannot understand the Conservative Party, its recent travails and its future triumphs, without acknowledging the existence of what I call ‘the party in the media’ – the editors, commentators, and journalists who have a huge impact on Tory strategy, or whatever passes for it.
Finally, it is worth taking notice of another group of people who are paid to observe politics – the academics – just to see if they have anything to say which might give us a few clues. As an academic myself, I am well aware of and sympathize with the standard criticisms made against us – that we live in ivory towers working up abstract theories which have little or no purchase on the real world, that we phrase our findings in language that obscures rather than illuminates, and that we seem to know more and more about less and less. But I also know that academics do have something to contribute if only we can communicate it in a way that readers who simply take an intelligent (and possibly an active) interest in politics can understand and even enjoy. Judging from high-street bookshops, historians have managed to do this. There is no real reason why political scientists – especially if we, too, can weave together narrative and analysis – cannot join them.
Academics are lucky people. We can get behind the headlines and beyond the collective memories and conventional narratives about parties that often disguise and distort what actually went on inside them. Academics can get at what people were thinking, saying, and doing at the time rather than what those same people hazily recollect. And, by treating politics as a science as much as an art, academics collect data about parties and elections all over the world, thereby discovering patterns and regularities which suggest that stuff that seems strange – the failure of politicians and parties to do whatever it takes to snatch and cling onto power, for example – can actually make perfect sense, at least to those directly involved. Knowing all this means that we stand a much better chance of producing a satisfying rather than a simplistic, silver-bullet-style explanation – an explanation which fits with how things work in the messy, nuanced, accident-prone, and incident-packed real world in which politicians (and, yes, even academics) try their best to live.
And that, when it comes down to it, is the purpose of this book. One of the nicest things said to me by a Tory politician as I was trying out my ideas on this topic was that I had ‘really got under the skin of the Conservative Party’. I don’t know if he was right. But what I do know is that only by doing that can we see where and why it went so wrong under and after Margaret Thatcher and how David Cameron put things right – or at least half-right.
The puzzle
Once upon a time, every book written about the Conservatives contained an obligatory reference to what one expert calls ‘the party’s quite remarkable facility for adaptation and, closely allied to this, its appetite for power, often indeed its readiness to subordinate all other considerations to that one objective’.1 The Tories, after all, could claim to be not just the world’s oldest political party but also one of its most successful. First emerging in the eighteenth century, they governed Britain for much of the nineteenth and most of the twentieth. Naturally, they suffered the occasional reverse. But part of the key to their success in the twentieth century was their uncanny ability, following the loss of an election or (at most) two, to pick themselves up, dust themselves off, and start all over again. In short, the Tories could claim to be the country’s ‘natural party of government’ because, whenever they found themselves in opposition, they rapidly managed to do whatever it took to get back into office.
Precisely what that was is actually no mystery. Without forgetting the wisdom of the old saw that ‘governments lose elections, oppositions don’t win them’, the party out of office essentially has to present itself as a convincing alternative. In the words of one investigation into how the Tories traditionally managed to recover power, this normally means the opposition is able to demonstrate that ‘it has a credible leader, is united, . . . and has policies which are not unwelcome to the electorate and which have enough coherence and content to be sustainable against attack’. Consequently, the same study concluded, there are five ways in which the Party normally places itself in a position to win next time around.
The first of these is ‘fresh faces’: a new leader or leadership team, and especially the sense of a change of generations. The second is ‘cohesion’: the maintenance of unity and discipline within the party, which is essential to convey a sense of purpose and effectiveness. The third is ‘visibility’: a new agenda or a distinctive position, and a distancing from past unpopular policies and their legacy. Here it is important to have an impact upon the political elite and opinion formers, in order to give credibility to revival and reorientation, and for this to be communicated to a wider audience. The fourth element links to this, and is ‘efficiency’: not just an improved or revived party organization, but the sense that the party is at least master in its own house and can respond with speed and authority when the need arises. The final element is ‘adaptability’: a hunger for office, and a pragmatic or unideological approach which gives room to manoeuvre and seize the openings that appear.2
When the Tories ran into trouble at the end of a decade of success during which Margaret Thatcher won three elections on the trot, they failed to do these five things – or certainly failed to do them all simultaneously – until at least 2005, when David Cameron was elected leader.
Some of Cameron’s predecessors over the previous 15 years (John Major, William Hague, Iain Duncan Smith, and Michael Howard) could claim, more or less convincingly, to be a fresh face. But all of them, bar one, led a fractious party and none of them managed to convey a sense that the Conservatives were really moving on and coming together organizationally. As for elevating pragmatism over ideology, forget it. Inasmuch as they did so at all, they were left looking not realistic but opportunistic. For the most part, however, they did nothing of the kind, focusing instead on policies which made them look, at best, out of touch and old-fashioned and, at worst, mean-spirited and obsessive.
This might not have mattered so much had the Tories been facing the same Labour Party they had managed to beat so easily in 1979, 1983, and 1987 – or indeed the Labour Party led by Ed Miliband and then Jeremy Corbyn.3 But they were not. Labour had been making progress, at first halting and then headlong, towards what it continually and persuasively claimed was the centre ground of British politics.4 As a result, first in opposition under Neil Kinnock and John Smith and then in government under Tony Blair, Labour managed to tap into the widespread belief among the British that there need not be – indeed should not be – a trade-off between social justice and economic growth, between fairness and efficiency, between quality public provision and higher net disposable incomes. At the same time, it also managed to persuade people that a trade-off does exist between lower taxes and investment in the NHS and state education, which, along with economic well-being and the control of crime (and immigration), remain at the top of voters’ lists of things governments are supposed to deliver. In so doing, Blair effectively neutralized – at least until 2007 – the classic Conservative argument, deployed to devastating effect since 2010 by George Osborne, that Labour’s heart might have been in the right place but that it couldn’t run the economy. ‘New Labour’ also made any claim by the Tories that they could simultaneously deliver lower taxes and maintain public services look like a pathetically transparent attempt to have their cake and eat it too. That Conservative politicians between 1997 and 2005 – when they weren’t too busy talking up threats to national identity and law and order – continued to make that claim suggests that they, to some extent at least, were the authors of their own misfortune.
Doing things differently would probably not ha...