The Black Door

Spies, Secret Intelligence and British Prime Ministers

Richard Aldrich, Rory Cormac

• Publisher: William Collins
• Year: 2016
• ISBN:
  9780007555451
• Topic: Política y relaciones internacionales
• Subtopic: Libertad política

The Black Door explores the evolving relationship between successive British prime ministers and the intelligence agencies, from Asquith’s Secret Service Bureau to Cameron’s National Security Council.

PART ONE

CREATING SECRET SERVICE

1

Herbert Asquith, David Lloyd George and Andrew Bonar Law (1908–1923)

A rumour spread next day in true war-time fashion that Lord Chetwynd had caught three German spies trying to signal the Zeppelin with lights, and had shot them out of hand.

David Lloyd George1

The British intelligence system was already immense by the dawn of the twentieth century. The entire British Empire, sprawling across more than half the globe, served as an intelligence machine – a veritable ‘empire of intelligence’.2 It depended on information of every kind to project British rule in remote places. Romanticised in the works of Rudyard Kipling, intelligence was about amateur imperial adventurers, revelling in deceit and subterfuge. Unlike today, espionage was conducted informally by local agents, eccentric travellers and scholars. Lawrence of Arabia and the legendary team of British archaeologist-spies at Carchemish, working across the Turkey­–Syria border, typified the way great-power rivalry in the Middle East and Asia mixed with academic debates about how archaeology might change understandings of the Bible. Importantly, scholarly research was not merely a cover for secret service work. Intelligence was considered to be every kind of information, including that obtained by peculiar scholars who spoke many languages, collected eggs, bulbs and butterflies, and mixed ‘comfortably and innocently’ with the local population. Indeed, at the turn of the twentieth century, British officials in Mesopotamia feared an archaeologist-gap and urged the India Office to contact learned societies which could send more scholar-spies to buttress British interests.3 India itself, the vast jewel of the empire, depended on spies too. It became an extraordinary ‘empire of information’, allowing just over a thousand British civil servants to superintend close to 280 million people.4

An empire of information, in some form, existed at home too. New public health initiatives and social programmes meant the state knew much more about its people than ever before. Meanwhile Irish nationalists and bomb-throwing anarchists from the Continent had forced the Metropolitan Police to develop the world’s first ‘Special Branch’ in 1883. Inevitably perhaps, by the first decade of the twentieth century, Special Branch had broadened its net beyond Fenians and anarchists to collect intelligence on possible foreign spies, anti-colonial agitators, suffragettes, union leaders, pacifists, and even those with radical views on sex.5

Nevertheless, these developments were more about keeping authoritarian surveillance at arm’s length from the British public. Mysterious battles with Russian spies in Central Asia were reassuringly remote, while the creation of a small Special Branch of intrepid detectives to counter terrorism was seen as a low-key alternative to the sort of harsh and repressive security legislation preferred in Europe. Accordingly, intelligence remained a matter for distant vice-consuls travelling to Samarkand, or detective sergeants in Whitechapel. It rarely resonated at the centre of British government. More rarely still did a prime minister take a personal interest in espionage.

All this began to change over the next decade. Bizarrely, it was literary fashion which drove the transformation of British intelligence, thrusting it towards the heart of government for the first time. A wave of popular Edwardian ‘invasion literature’ forced Herbert Asquith, an otherwise uninterested Liberal prime minister more focused on progressive domestic reforms, to take notice of foreign espionage. Acting only under pressure, Asquith set in train a course which would fundamentally alter the landscape of British intelligence forever. The intelligence revolution may have been forged by Churchill and Attlee, but the first sparks came through the unlikely Asquith. Nonetheless, despite the expansion of espionage during the First World War and greater interest on the part of Asquith’s successor David Lloyd George, it took time and endless trouble before successive prime ministers learned to use intelligence effectively.

Fiction paved the long pathway to the creation of a modern British intelligence service. During the nineteenth century, a stream of often sensational crime novels fed a growing public fascination with detection, police work and science. This generated cultural change, gradually eroding anxieties about government surveillance. Created by writers as diverse as Edgar Allan Poe and Arthur Conan Doyle, fictional detectives gradually became heroic figures to be worshipped, rather than bogeymen associated with distasteful and un-British authoritarianism.6 Journalists, who often exchanged information with real policemen, accelerated this transformation, helping to create real-life detectives who enjoyed a similar cult status.7 William Melville, who became head of Special Branch, was one example. He famously foiled several anarchist attacks, including the ‘Jubilee plot’ against Queen Victoria in 1887 and another dastardly scheme known as the ‘Walsall plot’, in which anarchists from the West Midlands sought to manufacture a bomb. Many now believe that he created the latter plot himself for mere self-glorification.8

Over three hundred spy novels went into print between 1901 and 1914, mostly focused on the ‘German Menace’. This new fiction effectively rebranded itself as a barely concealed form of ‘true crime’ writing, supposedly based on the patriotic leaking of government secrets. In 1903, Erskine Childers published The Riddle of the Sands, which narrated the summer sailing adventures of two young British men along the East Frisian coast who stumble upon a German plot to invade England with a flotilla of barges. Childers claimed to be offering the British people nothing less than a warning of what was to come, and Rudyard Kipling urged the public to support him in taking a firm stand against the ‘shameless Hun’.9 The story was improbable – the German admiralty had long written off the possibility of an invasion from East Frisia – but the hapless Royal Navy was forced to investigate the claims regardless.10

Members of Parliament from constituencies on the east coast, the closest part of Britain to Germany, bombarded the First Lord of the Admiralty, Lord Selborne, with difficult questions. In response, he ordered the Naval Intelligence Division to write a detailed report on the invasion plan outlined by Childers. After reconnaissance of the Frisian coast, the report concluded that the lack of railways and roads, together with the shallowness of the water and general lack of facilities, made a secret invasion from there impossible. ‘As a novel it is excellent; as a war plan it is rubbish,’ insisted Lord Louis Battenberg, the director of naval intelligence. To Childers’ barely disguised delight, the Kaiser banned the book in Germany. Childers also claimed that when he next went sailing in the Baltic, German spies dogged his every move.11

Spy fiction developed a darker side, shifting its focus closer to home, portraying immigrants and foreign visitors as the hidden hand of subversion. The new trope was Germans as a hidden fifth column already within Britain. Most active of all was a mischievous thriller-writer called William le Queux. He almost single-handedly created a fifth-column panic and then demanded the creation of a domestic security service to combat the undercover ‘German Menace’, with which the day of reckoning would surely come.12 Le Queux’s The Invasion of 1910 became the most influential of these books. Published in 1906, it told of German armies over-running the country with the help of spies and saboteurs. Le Queux admitted to a political purpose, explaining his intention to illustrate how poorly British defences would stand up to this sort of sneak attack. He also insisted that the content of his book was factually correct, and had been informed by conversations with ‘the authorities’.13

William le Queux posed as an international espionage expert, but was in fact nothing of the sort. Instead, he was a tireless and well-paid pulp-fiction writer who produced five novels a year until his death in 1927.14 The Invasion of 1910 sold more than a million copies, and publicists at the Daily Mail, which possessed serialisation rights, sent columns of men marching up Oxford Street dressed in Prussian uniforms, complete with bloodstained gloves, carrying sandwich boards that advertised the book.15 An inevitable flood of literary emulators hit the presses as le Queux’s royalties rolled in. Not to be outdone, le Queux responded in 1909 with Spies of the Kaiser, which insisted that no fewer than 5,000 German undercover operatives were at work in Britain. This was a new idea. A whole undercover army, not just a few spies and saboteurs, were apparently biding their time until the Fatherland told them to retrieve their weapons from a series of arms dumps in the British countryside. No less important, the novel also claimed to offer the inside story of intrepid British detectives working with agents to uncover these foreign networks.16

The British public were whipped into a frenzy. Politicians who wanted to expand Britain’s relatively small peacetime army jumped on the bandwagon. In 1908 Lord Roberts, Britain’s most distinguished soldier, claimed there were 80,000 trained undercover German soldiers in England ready to assist in the event of an invasion.17 Newspapers offered £10 to members of the public who reported sightings of suspicious activities that they could pass on to le Queux so he could ‘supplement his investigations’. Unsurprisingly, they were inundated with information, and soon it seemed a supposed German saboteur had been spotted in every town in the land,18 including some unfortunate Foreign Office clerical staff holidaying on the east coast.19 The government was annoyed by the obvious political scheming by advocates of increased arms spending. In 1906, Asquith’s predecessor as prime minister, Henry Campbell-Bannerman, rose to his feet in Parliament to denounce le Queux as a ‘pernicious scaremonger’ whose stories risked provoking an unnecessary war with Germany.20

Despite prime ministerial condemnation, several key figures in government worked hand in glove with le Queux. James Edmonds, head of a fledgling military intelligence unit called MO(5), maintained that Berlin had an extensive ring of operatives in Britain. Edmonds had long been nagging a dismissive Richard Haldane, secretary of state for war under both Campbell-Bannerman and Asquith, about the shortcomings of British counter-espionage. He therefore found le Queux’s activities more than welcome.21 Indeed, much of the evidence that he presented to his masters came from le Queux, who in turn said that he had received it from concerned members of the public. There had only been five cases of German espionage reported in 1907, but unsurprisingly by 1908 this had risen to forty-eight. Edmonds created a helpful map of agent sightings. Perhaps equally unsurprisingly, this was soon leaked to the press. He also recruited William Melville, the former head of Special Branch, who while a talented agent-runner, had a reputation for embellishing his stories.22

By 1909, nefarious German agents were apparently hiding behind almost every bush in Britain. Asquith, however, had other things on his mind. His overwhelming, and rather challenging, priority was to take on and reform the House of Lords, which had consistently blocked his progressive agenda. The time and energy he devoted to this issue only increased further when the Lords famously rejected his government’s budget in 1909. But with the public suffering from spy-fever, the prime minister now had to take note of intelligence matters. As chancellor of the exchequer and in his early months as prime minister, Asquith had already chaired a senior committee to consider the invasion threat in response to pressure from Roberts and le Queux. His report demolished their theories, and showed a surprise attack to be impossible.23 Nonetheless, still under immense public pressure, in 1909 Asquith asked the Committee of Imperial Defence to consider the danger of foreign agents operating in the UK. All this accelerated an important cultural change away from the idea that counter-espionage at home was authoritarian and un-British.24

Although flawed and bogus intelligence reports shaped its ultimate findings, Asquith swiftly sanctioned the committee’s recommendation to establish a new Secret Service Bureau. This top-secret decision – very few people knew about the bureau’s existence – fundamentally altered the landscape of British intelligence forever. British espionage was nothing new, but its formalisation and growing proximity to Downing Street marked the beginning of a more organised and centralised intelligence system. It was not yet, though, an intelligence community linked to Downing Street, and early prime ministers remained unable or unwilling to engage with the secret world particularly closely.

The Secret Service Bureau was comprised of two branches. MO5(g) was the domestic branch, responsible for counter-espionage, and would soon come to be known as ‘MI5’, or the security service. It inherited a number of army counter-intelligence officers, and was led by Captain Vernon Kell, deputy to James Edmonds at MO(5). Kell went on to serve continuously in the role for more than twenty years. Ever present around intelligence matters for nearly half a century, Winston Churchill, as home secretary, oversaw Kell’s appointment. Adding a symmetry to Kell’s long career as spymaster, Churchill, when prime minister, would remove him from office in 1940.25 The Secret Service Bureau also had a foreign branch. Initially called MI1c, it soon restyled itself MI6, and was headed by the remarkable figure of Sir Mansfield Cumming, a retired naval officer. Despite losing his leg in a traffic accident in France, Cumming continued to run MI6, speeding down the corridors of Whitehall by planting his wooden leg firmly on a modified child’s scooter. MI5 was a small organisation and MI6 even smaller, but a key change had occurred. Hitherto, individual government departments, especially the India Office, had gathered intelligence locally and conducted espionage for their own purposes. Now Whitehall had an interdepartmental intelligence machine at the centre, delivering a ‘service’ to all government departments and anointed with the cult of specialness.26

After 1909, Asquith’s government went further. Harassed by continued public concern about nefarious German activities, it introduced the trappings of a secret state that previous successive governments had resisted for a century. Between 1909 and 1911, Asquith not only created the Secret Service Bureau, but also passed a draconian Official Secrets Act and allowed for much wider mail interception, something a Liberal government had banned as a diabolical infringement of liberties some fifty years before.27 The new Official Secrets Act, encouraged by Edmonds of MO(5), also helped the government to crack down on the press.28 Asquith, a facilitator rather than a dictatorial prime minister, also set up a new committee under Winston Churchill to consider how Britain would tackle ‘Aliens in Time of War’. With foreigners and exiles at the heart of this scare, the government created a register of aliens living in Britain. By 1913, it would have 11,000 Germans on this list, and had already prepared internment legislation.29 Notwithstanding domestic spy-fever, on the eve of the First World War much British intelligence activity lay elsewhere. Intrepid consuls, attachés and military officers carried out most of the spying on German armament programmes, while MI6 relied on the collection of specialist journals by travellers, or open observations, rather than actual espionage. Despite a lack of coordinated assessment, this material gradually filtered up into the higher reaches of government and influenc...