Category Archives: Technology

HASP: Spycatcher’s Last Gasp

Peter Wright

(This report, on the dubious testimony of Peter Wright, the author of Spycatcher, concerning Agent Sonia and her wireless transmissions, is a long and challenging one, and I issue my customary health warning: Do not read this if you are of a sensitive disposition, or while operating agricultural machinery. I decided to lay out every step of my reasoning, with references, as I believe that, with the delivery of the authorised History of GCHQ in a few months’ time, it is important to present a comprehensive story of the slice of wartime Soviet wireless traffic that Wright focused on in his book. The interest in Spycatcher indicates that a mass of persons are fascinated by this topic: questions about possible traitors in the midst of the Security Service do not go away. I believe the issuance of this report is especially timely, as the recent feature in the Mail on Sunday should intensify the interest in the case that Wright made against Sonia and her alleged protector, Roger Hollis. If any of my readers would prefer to work with a Word version of this bulletin, in the belief that they might want to pore over it, and annotate it, please contact me at antonypercy@aol.com.  After a thorough background check by my team of ultra-sensitive, highly-trained, Moscow-based security personnel, the report will be sent to you.)

“Stella Rimington and some friends in the Security Service called Wright ‘the KGB illegal’, because, with his appearance and his lisp we could imagine that he was really a KGB officer.”                                                                                                         (Defending The Realm, p 518)

“I want to prove that Hollis was a spy; if I can do that I will be happy.” (Peter Wright to Malcolm Turnbull, from the latter’s ‘Spycatcher Trial’, p 31)

“The time has come for there to be an openness about the secret world of so long ago … the consequences of Hollis being a spy are enormous. Not only does it mean that MI5 is probably still staffed by people with similar view to him, but it means that ASIO was established on terms with the advice of a Russian spy.” (Peter Wright in the witness-box, Sydney, December 1986)

Contents:

  1. Peter Wright and ‘Spycatcher’
  2. The Background
  3. Cable or Wireless?
  4. War and Peace
  5. VENONA and HASP
  6. Wright on HASP
  7. The Remaining Questions
  8. The Drought of 1942-44
  9. Why did Wright Mangle the Story so much?
  10. Conclusions

Peter Wright and ‘Spycatcher’

As an ex-IBMer (1969-1973), until I read Spycatcher in the late nineteen-eighties, the only ‘HASP’ I knew was the Houston Automated Spooling Priority program (about which I shall mercifully write no more). One of the major contributions to mole-hunting that Peter Wright believed he made, in his best-selling account of dodgy business within MI5, was the unveiling of a new source of electronic intelligence, namely (as he described it) ‘the wartime traffic stored by the Swedish authorities known as HASP’. By citing a previously unknown and ever since unrevealed message that purported to indicate the size of Sonia’s ‘network’ of spies in 1941, Wright’s assertion has exerted quite a considerable influence on the mythology of Soviet ‘superspy’ SONIA. If judged as credible, his testimony boosts her achievements in England even beyond what the woman claimed in her memoir, Sonya’s Report. Moreover, Wright used this discovery as a major reason for confirming his belief that Roger Hollis was the Soviet mole known as ELLI: he drew attention to this accusation in his presence in the witness-box during the Spycatcher trial, and thus the process by which he came to this conclusion is of profound significance.

Spycatcher sold over two million copies. This success was mainly due to the outcome of Her Majesty’s Government’s lawsuit against the author before publication, with Malcom Turnbull’s successful defence in the trial of 1986-87 issuing a stern blow to the forces of hypocritical secrecy. He was able to show that the British authorities had connived at, or even encouraged, the publication of Chapman Pincher’s two books, Their Trade is Treachery, and Too Secret Too Long (as well as Nigel West’s A Matter of Trust), which made nonsense of the claim that a ban on the whole of Spycatcher was necessary for security reasons. It was the obstinacy of Margaret Thatcher, abetted by poor advice, that caused the lawsuit to be pursued. The irony was that it was Wright who had fed Pincher most of his stories, and Pincher would later amplify Wright’s case against Hollis with the very influential Treachery. That is why this article is so important. Those two million-plus readers need to learn the facts about a critical part of Wright’s story.

The Background

Another significant outcome of a careful study of Wright’s claims concerning the HASP story is the uncovering of secrets about the interception and decryption of electronic traffic that the British intelligence services (MI5, MI6 and, especially, GCHQ) would rather the public remain ignorant of. The authorised histories of MI5 (Andrew) and MI6 (Jeffery) steered well clear of analysis of the mechanics of wartime electronic espionage, since these volumes were designed and controlled as organs of public relations. No discussion of Sonia, or the controversies surrounding illicit wireless in wartime Britain, can be found in their books, and Andrew (especially) points readers towards the secondary literature without any indication of how reliable it is, or how selectively it should be explored.  Moreover, I regret that I am not confident that all will be revealed to us when the authorised history of GCHQ (Behind the Enigma, by Professor John Ferris) is published later this year. While a subsidiary objective of my focus on Wright is thus to provide a more rigorous analysis of the often puzzling story of the Allied effort to interpret Soviet intelligence traffic in World War II, a more thorough account will have to wait until a later bulletin.

The secondary literature almost universally shows an alarming confusion about the techniques and technology that underlay the surveillance of the traffic of foreign powers before, during, and after WWII. The largely American literature on the VENONA program (to which HASP was a critical adjunct: see below) is distressingly weak on technology, and focuses almost exclusively on the interception of traffic in the United States. Even such a well-researched and methodical work as Philip H. J. Davies’s MI6 and the Machinery of Spying contains only two short references to VENONA, guiding the reader (note 32, p 237) for ‘a (contested) British version of the story’ to Peter Wright’s Spycatcher. This seems to me a gross abdication of critical responsibility. Davies concentrates of human ‘machinery’, not technology, and delegates coverage of problematic matters to a source he instantly characterizes as dubious. It would appear, therefore, that, even though Wright’s story does not derive from any published archive, his controversial memoir has become the default – but flawed – authority. Yet he was a minor officer in the grand scheme of things, and an elderly man with a grudge and a failing memory when his book was composed.

It is certainly difficult to obtain reliable confirmation of the essence of HASP from other academic, or pseudo-academic, sources. One might, for example, have expected to learn about it in Richard J. Aldrich’s 2010 work, GCHQ, yet, while providing a comprehensive chapter on HASP’s cousin VENONA, the author does not mention the term. The only other analyst who appears to have written explicitly about HASP without simply echoing Wright’s account is Nigel West, in his 2009 book Venona. West has overall provided a competent guidebook to the initial breakthroughs on decryption, and an excellent coverage of the content of VENONA traffic, with emphasis on the London-Moscow communications, although it would benefit from a revision to consider the relevance of such sources as the Vassiliev Notebooks (see https://digitalarchive.wilsoncenter.org/collection/86/vassiliev-notebooks). Venona is a highly readable summary for the curious student of intelligence, but West’s coverage of the mechanics of VENONA is spotty and inconsistent. Moreover, his representation of the HASP traffic is so different from that of Wright that I believe the topic merits greater scrutiny, and it is my goal here to provide that level of inspection, and assess the validity of what Wright claimed. This is uncharted and complex territory, however, and the landscape is strewn with pitfalls.

VENONA was one of the major successes of British-American co-operation on intelligence matters after WWII. Owing to a procedural mistake in 1943, a large number of GRU (military and naval intelligence) and NKVD/KGB (* state security) messages exchanged between Moscow and outlying stations in foreign embassies employed a defective technique for enciphering highly confidential messages – the re-use of so-called ‘one-time pads’. Intelligence agencies have regarded one-time pads as the most watertight way of preventing enemy decryption of messages, and they were adopted by the Soviet Union in the 1930s. (Many readers will be familiar with the concept if they have read Leo Marks’s Between Silk and Cyanide.) Alert cryptanalysts in the National Security Agency (NSA), inspecting messages in 1946, noticed unusual patterns, and in 1948 were joined by their British counterparts from GCHQ in exploring the phenomenon. By applying painstaking techniques to detect repeated sequences, they were able to initiate a project that gradually disclosed several networks of spies in the USA, Canada, Britain and Australia, leading to the successful prosecution of such as Julius Rosenberg, Klaus Fuchs, and Alan Nunn May, and the identification of Donald Maclean. VENONA was not formally revealed to the public until 1995.

Yet exactly what this ‘re-use’ entailed, and where and when it took place, and to which cryptological tools it applied, remains one of the most vexing puzzles in the VENONA story. It is as if the practitioners, when explaining their successes to the lay historians who carried their accounts to the world, wished to keep the process and sequence of events to themselves, as a defensive measure to protect their secrets, and maybe, even, to exaggerate what they were able to accomplish. A deep integrative history is sorely needed.

[* The naming of the Soviet Security Organization changed frequently. In 1934, the OGPU was transformed into the NKVD, which for a few months in 1941 became the NKGB, before reverting to NKVD until April 1943. In March 1946, it became the MGB, but foreign intelligence was transferred to the Committee for Information (KI) from October 1947 to November 1951. In March 1953, on Stalin’s death, the unit was combined with the MVD, out of which the KGB emerged, after Beria’s execution, in March 1954. Source: Christopher Andrew. I sometimes use ‘KGB’ in this article to refer to the permanent body, as do many authors.]

Cable or Wireless?

Eastern Telegraph Cables: 1901

One conundrum in the analysis of VENONA and HASP has endured: no author on the subject is precise about where and when VENONA (or HASP) was the result of intercepting cable traffic, and where and when it involved wireless traffic. This distinction is important when one considers the challenges facing the counter-espionage organisations of the nations trying to protect themselves. The term ‘cable’ is frequently used as a generic term for ‘telegram’, reflecting its historical background, but telegrams sent by wireless should definitely not be called ‘cables’. Christopher Andrew, in Defending the Realm, makes a useful distinction, but his account is incomplete and thus overall unsatisfying. He contrasts (on page 376) the regulations pertaining in the UK, where ‘even before the Soviet entry into the war, the Foreign Office had agreed that the Soviet embassy in London could communicate with Moscow by radio on set frequencies’, and adds that a project was soon underway to intercept these messages. On the other hand, no corresponding agreement existed in the USA, where, instead, ‘Soviet messages were written out for transmission by cable companies, which, in accordance with wartime censorship laws, supplied copies to the US authorities.’

This statement is probably an echo of what appears in the staff (but not ‘official’) story of VENONA, issued by the NSA/CIA in 1966 (VENONA: Soviet Espionage and the American Response, edited by Robert Louis Benson and Michael Warner). In the Preface (p xii) appear the following sentences: “Although Soviet intelligence services had clandestine radio transmitters in diplomatic missions located in several American cities, these apparently were to be used only in emergencies. In consequence, KGB and GRU stations cabled their important messages over commercial telegraph lines and sent bulky reports and documents – including most of the information acquired by agents – in diplomatic pouches.” This statement moves us closer to the truth, but in my opinion still misrepresents the essence of the Soviet strategy concerning clandestine systems, and does not explain whether these secret channels were intercepted at all.

Confusion abounds. For example, in the very first sentence of Venona, Nigel West writes of the project to intercept Japanese traffic in October 1942 as follows: “Cable 906 purported to be a routine circular in seven parts and, as it had come off the wireless circuit linking Tokyo to Berlin and Helsinki, it underwent the usual Allied scrutiny to see if it betrayed any information of strategic significance.” Cables cannot ‘come off’ (whatever that means) ‘wireless circuits’, and it is inaccurate to describe temporary wireless paths as ‘circuits’, since wireless transmission is by definition unconnected. It makes sense to refer to a ‘circuit’ linking ‘Tokyo to Berlin and Helsinki’ only in terms of a conceptual agreement about callsigns, frequencies, and schedules between intelligence services and outposts. As another example, the heading for the NSA’s official packaging of the London to Moscow traffic (at  https://www.wilsoncenter.org/sites/default/files/media/documents/article/Venona-London-GRU.pdf ) is titled ‘London GRU – Moscow Center Cables: Cables Decrypted by the National Security Administration’s Venona Project’, a regrettable misrepresentation of reality. The messages were sent by wireless.

The misconception is aggrandized by Peter Wright himself. In Spycatcher, the author, the self-professed expert in these matters, writes (p 182): “Whereas the Americans had all the Soviet radio traffic passing to and from the USA during and after the war, in Britain Churchill ordered all anti-Soviet intelligence work to cease during the wartime alliance, and GCHQ did not begin taking the traffic again until the very end of the war.” Sadly, every clause of this woeful sentence contains at least one blatant error, which casts serious doubt on his reliability on other matters. Specifically:

  1. The Soviet VENONA traffic to and from the USA was almost exclusively commercial cable traffic.
  2. ‘Had all the Soviet radio traffic’ is meaningless. Did the Americans intercept it all? Most certainly not. As other experts have pointed out, wireless traffic was banned (officially) during the war. The Soviets used wireless as an emergency back-up system, but also as a channel for clandestine espionage traffic.
  3. No one can point to the minute where Churchill ordered all interception, let alone all intelligence work, to cease. Hinsley’s famous footnote [see below] speaks only of ‘decryption and decoding’, not interception, and does not constitute an authoritative record. (Professor Glees reports conversations with Hinsley on this point in his book The Secrets of the Service: what Glees was told, namely that the Y Board may have issued such an order, now appears to be confirmed by the in-house history of the NSA.) We know that interception of signals continued, if erratically, throughout the war, and that Alastair Denniston, previously head of GC&CS, started his new project on Soviet traffic in late 1942.
  4. GCHQ did not come into existence until 1946. Before that the institution was known as GC&CS (Government Code & Cypher School). During the war, however, RSS was responsible for ‘taking the traffic’, and never reported to GC&CS. We know from RSS files that it monitored Soviet traffic, and that the ISCOT project started picking up Comintern messages in 1943.

Within this fog of misrepresentation a very important distinction remains. A cable is a wire, with the important corollary that those agencies that control the input to the physical cable may have special authority (or power) to intercept and store the traffic that is passed to them. Such transmissions can also be detected clandestinely by specialized sensory equipment, which would have to be laid close to the cable. Thus cables are a direct, bounded, targeted medium and not universally detectable. (Today’s fibre optic cables, which GCHQ and the NSA tap, follow largely the same oceanic paths used by the cables laid at the end of the nineteenth century.) Wireless traffic is looser: it is transmitted over the ether. It may be picked up by local groundwaves, or, remotely, by any receiving device that is geographically well-positioned to receive shortwave transmissions, allowing for the vagaries of atmospheric conditions, and frequencies used. Yet, while the atmosphere is lawless, the source of the transmission is frequently concealed, and the activity unpredictable. Wireless transmission presents a completely different set of security challenges.

P. S. I am grateful to Ian W. who, on the day this report was published, informed me that ‘cables’ might be transmitted for part of their journey over ‘wireless’ links – something I had suspected, but had not been able to verify. Ian also mentioned that, half a century ago, it was common for wireless contacts to be referred to as ‘circuits’.

War and Peace

Earlier in the century, circumstances – and improvements in technology – had encouraged the use of wireless as a medium for confidential traffic. Private or nationally-owned cable facilities had been shown to be liable to attack and destruction. Such sabotage happened when the British cut Germany’s nationally-owned transatlantic cables in 1914, an event that forced German diplomatic traffic to be routed through ‘neutral’ third parties. Britain used its sway to intercept German traffic, and with cryptological skills abetted by the provision of codebooks supplied by the Russians, started deciphering German messages. In February 1917, the British deciphered the Zimmermann Telegram, which had encouraged Mexico to join forces against the United States. When Zimmermann admitted the truth behind the cable telegram, public disgust brought the USA into the war.

Such an exposure encouraged experimentation with a rapidly developing wireless technology. (In Spycatcher, Peter Wright himself explained how, after World War I, his father assisted Marconi in convincing the British government that the beaming of short-wave wireless signals would be more effective than deploying long-wave technology as a means of linking the Empire.) In turn, as practices and understanding matured, that led to the important adoption of water-tight encryption mechanisms. Correspondingly, in the next two decades of peace, host governments tried to monitor such processes that originated on their home territory, by attempting to pick up open transmissions from the air, to set about decrypting them, and thus identifying possible hostile threats. The British project known as MASK, which detected Comintern traffic in London in the mid-thirties, was an example of such.

The advent of war, however, made a more spirited approach to trapping and prosecuting illicit wireless transmissions much more urgent. For example, at the outset of World War II, the British were fearful of the possibility of swarms of enemy wireless operators in their midst. They were initially not so scared about routine intelligence-gathering as they were about the (imaginary) menace of such spies using wireless to guide German bombers to their targets. The government also wanted to control the dissemination overseas of secret intelligence by conventional agencies. It made demands to foreign embassies and legations about being informed of wireless frequencies, and even call-signs, before giving approval for their use. Since a tacit understanding about reciprocal needs existed, governments often turned a blind eye to some technical breaches (such as the British with the Soviets, and the Swiss with the British). To monitor abuse of the airwaves, interception services then had to deploy enhanced wireless detection mechanisms to collect such clandestine messages, and maybe direction-finding/location-finding systems and vehicles to verify the source of such messages (as happened with the Soviet Embassy in London in 1942.) The elimination of any possibly overlooked German wireless agents was critical for the success of the Double-Cross system.

The UK government thus permitted the use of wireless transmitters on embassy premises only for Allies, while allowing, as a special case, the Polish and Czechoslovak governments-in-exile to have their own independent wireless stations, the Czech station in Woldingham, Surrey playing a very significant role. In the UK, all represented governments (including those in exile) clearly had a preference for using wireless rather than cable, in the belief that the traffic might not be picked up at all, and thus be more secure. The Soviet Union was in a unique position, as it was officially neither ally nor enemy from September 1939 until June 1941, but was hardly neutral, as it had, in that period been in a pact with Nazi Germany, and had aided the latter’s war effort against Great Britain. In those circumstances, it was supposed to use its wireless apparatus in the Embassy for diplomatic traffic only, and was instructed to inform His Majesty’s Government of frequencies and callsigns being used.   

Thus, when any embassy or legation in World War II wanted to send a ‘telegram’, it still maintained some level of choice. First, it had to deal with the local government, consider the regulations, and assess how strictly the rules were going to be enforced. Indeed, many such messages were enciphered, but still sent over private circuits. Copies were frequently taken by the local authorities, especially by those who (as with the USA) forbad the use of clandestine wireless by foreign governments. Indeed (as Romerstein and Breindel remind us in The Venona Secrets), in 1943 the US Federal Communications Commission detected illicit radio signals coming from the Soviet consulates in New York and San Francisco, and confiscated the apparatus. Consequently, the NKVD and GRU in the USA had to rely almost exclusively on commercial telegraph agencies to send their messages to Moscow. Likewise, all confidential traffic beyond the diplomatic bag that was sent back to Moscow by the embassy in Canberra, Australia (a vital VENONA source), was officially transmitted by commercial cable companies.

Romerstein’s and Breindel’s account corresponds in general with what NSA officers have written. Their statement is an echo of what appears in Benson’s and Warner’s history mentioned above. In that work’s Preface (p xii) appear the following sentences: “Although Soviet intelligence services had clandestine radio transmitters in diplomatic missions located in several American cities, these apparently were to be used only in emergencies. In consequence, KGB and GRU stations cabled their important messages over commercial telegraph lines and sent bulky reports and documents – including most of the information acquired by agents – in diplomatic pouches.”

Yet the FBI offers an intriguing twist to this story. In the archive of that institution (‘The Vault’) can be found some provocative assertions. An undated memorandum outlining considerations in using VENONA information in prosecutions (p 63) declares that ‘these Soviet messages are made up of telegrams and cables and radio messages sent between Soviet intelligence operators in the United States and Moscow.” While that is an implausible triad (cables and radio messages are both ‘telegrams’), it suggests a more complicated situation. And, on page 72, the writer measures, with some timidity, some political considerations, indicating that the Soviet Union might react in a hostile fashion to the news that the USA had been spying on its wartime ally, thus not acting ‘in good faith’. He writes: “ . . .  while no written record has been located in Bureau files to verify this it has been stated by NSA officials that during the war Soviet diplomats in the U.S. were granted permission to use Army radio facilities at the Pentagon to send messages to Moscow. It has been stated that President Roosevelt granted this permission and accompanied it with the promise to the Soviets that their messages would not be intercepted or interfered with by U.S. authorities.”

One can imagine the frequently naïve Roosevelt making an offer like this, but it is difficult to imagine that the wary Russians would take such an offer at face value, and have their cypher-clerks trek over to the Pentagon to send their material in the knowledge that it would probably be intercepted. Moreover, not all their traffic derived from Washington: New York and San Francisco were busy outlets. The item is undated, and apparently unconfirmed, and thus needs to be recorded as a footnote of questionable significance.

On the other hand, what is certain is that the Soviet Embassy in London breached the rules, even before Barbarossa, first of all by sending not just diplomatic traffic but also military and intelligence reports to Moscow on the acknowledged channels. Yet Soviet Military Intelligence (the GRU), which was for a while the only functioning intelligence unit in the Soviet Embassy, as the NKVD officers had reputedly been recalled for almost all of 1940, went far beyond what was permitted in order to deceive surveillance mechanisms. I refer to a VENONA message of July 17, 1940, from London to Moscow, which is titled ‘Setting up an illicit radio in the Soviet Embassy’. It unambiguously refers to apparatus sent over in the diplomatic bag, but without clear instructions, and requests more guidance on setting up the antenna. The GRU in London was trying to establish an alternative mechanism for transmission without informing its hosts, and, when the GRU rather absurdly suddenly were about to run out of one-time pads in August/September 1940, messages at that time specify that the ‘emergency system’ should be used. The emergency system was planned not just as a back-up procedure using a book-directed system for creating random keys (in place of the printed one-time pads), but as the deployment of an alternative wireless transmitter/receiver apparatus. (I analyse this phenomenon in more detail at the end of this report.)

To summarize, in the context of World War II: the pressures on combatants to prevent unauthorised intelligence from leaving the nation were intense. The distinction between the media was very important, as cables were finite, self-contained, and asynchronous, and could easily be collected by the host country. Wireless messages, on the other hand, were open, unconstrained, and always somewhat speculative, but required a sophisticated infrastructure just to be intercepted. Synchronicity was the goal with wireless, but was not always achieved: your target might not pick up your message and acknowledge it, or might receive it only partially. On the other hand, an unintended bystander might intercept it. Moreover, to circumvent the efforts of the authorities, units wanting to send intelligence back to their controllers would sometimes set up alternative wireless systems in secret, of which the local government had not been notified. I do not believe any analyst of VENONA has explained in detail how the respective traffic was transmitted or collected in each country, i.e. by cable, by authorised wireless, or by unauthorised wireless. Certainly, the experience – and opportunity – differed greatly for the British and American authorities.

VENONA & HASP

This confusion appears to have leaked into the VENONA-HASP muddle. In order to put the HASP phenomenon into the context of VENONA, I shall soon turn to the texts of Peter Wright, the primary source about HASP, and add detailed commentary on each passage. One of the difficult concepts to bear in mind with VERONA and HASP is that decryption (with the exception of the Australian intercepts) did not happen in real time. We are thus dealing with a process that attempted to decrypt messages that may have been transmitted two or three decades earlier, which were intercepted and stored at the time, but represent only a small percentage of the total messages that could have been theoretically available. Thus discontinuities and gaps are par for the course. Moreover, it is important to understand that the Soviets did not realise for several years that their systems had been exposed, and consequently did not rush to fix the problem. The fact of the breakthrough was revealed to the Soviets by the spies William Weisband and Kim Philby in 1949. Only then did the Soviets change their procedures, but they could do nothing about the historical traffic of 1940-48.

VENONA itself is a murky project filled with anomalies and unanswered questions, beyond the scope of analysis in this article. The set of facts that need to be borne in mind when considering HASP are the following:

  1. The key years of 1945 (when the damaged Soviet codebook gained at Petsamo was acquired by the USA, and when the GRU cypher-clerk Igor Gouzenko defected in Canada); 1946 (when Meredith Gardner made the first major VENONA decryption); 1949 (when ex-Comintern wireless operator Alexander Foote revealed GRU techniques in Handbook for Spies); 1954 (when Vladimir and Evdokia Petrov, Soviet cypher experts who had worked in Stockholm, defected in Australia); and 1959 (when the Swedes handed over HASP, the result of their decryption successes, to GCHQ and NSA).
  2. The GRU developed an auxiliary clandestine system to maintain secrecy. This consisted of a) an alternative method of using a secure one-time pad exploiting a reference book known to both parties (which could be used on the regular channel), and b) a separate wireless receiver-transmitter and protocols, not to be announced to the domestic authorities.
  3. In the USA and in Australia, the Soviet units used commercial cable channels almost exclusively. In Britain, all traffic was sent by wireless.

Wright on HASP

In 1987, Peter Wright (with the assistance of the journalist Paul Greengrass) published his best-selling work Spycatcher, an account of the efforts by the so-called ‘FLUENCY’ committee to identify a suspected mole in the senior ranks of MI5. Wright, who had been ‘chief technical officer’ within the service, was appointed chairman of the committee when it was set up in 1964. Because of the way the programme had unmasked figures such as Fuchs and Maclean, the disclosures from the VENONA project were viewed as possibly important providers of further breakthroughs. Yet successes with VENONA traffic had been slowing down in the early 1950s, and Wright stated that the project had come to a halt in 1954. A few years later a fresh injection gave the project new life. I do not intend to discuss the broader issues explored in Spycatcher: my focus is on a strict analysis of the passages where Wright writes about HASP.

Pp 185-187 [i] “In 1959, a new discovery was made which resuscitated VENONA again. GCHQ discovered that the Swedish Signals Intelligence Service had taken and stored a considerable amount of new wartime traffic, including some GRU radio messages sent to and from London during the early years of the war. “

Wright appears confused from the outset. He explicitly states that this traffic included messages that could be classified as ‘GRU’ and ‘radio’. But if this traffic had been stored, but not decrypted, how did the Swedish Service, or the receiving agency, GCHQ, know they were GRU exchanges until they were decrypted? Moreover, Wright states that these were radio messages sent ‘to and from London’. Does that mean between London and Stockholm or between London and Moscow? The suggestion could conceivably be the latter, as Stockholm would have been geographically well-situated to pick up messages targeted at Moscow, and there would be little reason for the GRU station in London to communicate with its Swedish counterpart (although a few such messages do exist in the archive). Why the Swedes would be interested, however, in intercepting and storing traffic that did not concern them directly is a puzzle in its own way. As an added complication, Fred. B. Wrixon, in his Codes, Ciphers & Other Cryptic & Clandestine Communications, states that the Swedes ‘had intercepted some GRU radio exchanges between agents [sic: my italics] in Great Britain and their headquarters in the Soviet Union’, (p 118), and that GCHQ gave the name HASP to the project to decipher them. Wrixon’s source is not stated. How Wrixon derived this information is not clear, but it eerily echoes one of Wright’s more outlandish caprices.

Did Wright mislead his readers, whether intentionally or not? I think so. His assertion about the nature of the traffic appears to be contradicted by Nigel West, who, in Venona, on page 120, presents an alternative explanation. He writes: “ . . . in 1959 the Swedish National Defence Radio Institute (Forsvarets Radioanstalt, FRA,) revealed that it had retained copies of a vast quantity of the Stockholm-Moscow traffic and negotiated with GCHQ to release its archive to the NSA via Cheltenham. This was the batch of intercepts codenamed HASP, and, bearing in mind that some of these texts had been encoded and signed by Petrov, there must have been a great temptation to confront him with them – if only to tax his memory by seeking clues to the missing, unrecovered groups.” West further explains that when the HASP material became available, ‘two 1945 VENONA intercepts from the Stockholm embassy, dated 16 July and 21 September, showed that Petrov, then codenamed SEAMAN, had been the personal cipher-clerk to two rezidents, first Mrs Yartseva, then Vasili F. Razin. However, their experience in Sweden had not prepared the Petrovs for the atmosphere of intrigue in Canberra.”

Thus West makes a very clear connection between traffic obtained locally in Sweden and the defection of Petrov and his wife in April 1954, and suggests, moreover, that HASP material was exclusively Stockholm-Moscow traffic. This is markedly in contrast to Wright’s representation. Yet West does not explain what the relationship was between the HASP and the VENONA material, how the former helped the GCHQ cryptanalysts, or where he derived his information. He refers to intercepts, but were these raw encrypted data, or partially decrypted texts – or both? The logic is very elusive, since the HASP messages are not separately identifiable, but it would appear that additional information enabled the cryptonym MORYAK (SEAMAN), as a key member of the Soviet embassy in Stockholm, to be identified as Petrov. And indeed, the source telegrams confirm Petrov’s statements from the memoir that he and his wife published in 1956.

The message of July 16 can be seen at: https://www.nsa.gov/Portals/70/documents/news-features/declassified-documents/venona/dated/1945/16jul_cipher_text_seaman.pdf, but the VENONA records of September 21 appear to contain no Moscow-Stockholm traffic. Nevertheless, the identity of SEAMAN can be confirmed from earlier traffic from Stockholm to Moscow, when Petrov was working in Moscow (see telegrams No. 797, of September 6, 1941, and No. 821, of April 30, 1942), before the Petrovs’ dramatic seven-month journey to Stockholm, via Siberia, South Africa, and Great Britain.

A significant distinction between the respective descriptions of HASP by Wright and West can thus be seen, with West, to support his cause, providing more tangible evidence of what the traffic contained. The account of another historian, Christopher Andrew, would appear to reinforce West’s description, although without actually mentioning HASP. On page 380 of Defend the Realm, Andrew writes: “Following requests during 1960, the Swedes supplied copies of wartime GRU telegrams exchanged between Moscow and the Stockholm residency, some of which were discovered to have employed the same one-time pads used in hitherto unbroken traffic with London. One hundred and seventy-eight GRU messages from the period March 1940 to April 1942 were successfully decrypted in whole or part.” Andrew’s message is explicit: these messages were not London-Stockholm traffic, but Stockholm-Moscow messages that the Swedes had apparently enjoyed some success in decrypting. His log of successful decryption applies to London-Moscow traffic, however, the suggestion being that both sets of traffic used the same one-time pads, and that no progress had been made by GCHQ on the London messages beforehand.

Moreover, what does that strange, anonymous notion behind ‘requests’ indicate? How did the ‘requestor’ learn about them? What was the crypto-analytical expertise of the Swedes, and had they previously shared experiences with GCHQ and NSA? The certain implication here is that the FRA had successfully deciphered some local GRU traffic, as West informed us. Yet it was not the messages themselves that were of relevance to GCHQ’s investigations, but a suggestion that the process of using stale one-time pads had been deployed, and that the revelations from these could be applied to traffic that the GCHQ possessed, but had been unable to break. This insight from Andrew (the source is the typically useless ‘Secret Service Archives’ from the authorised ‘historian’), and his immediately following comments, will turn out to be critical in working out what happened. It should also be noted that Andrew specifically contradicts Wright’s description of the essence of HASP, yet, with characteristic unscholarliness, includes Spycatcher in his bibliography.

Andrew’s failure to specify explicitly whether these one-time pads were the conventional set of random numbers created and printed by the KGB, or the alternative ‘reference-book’ mechanism used as a back-up system, is a critical oversight. I note also that this notion of ‘re-use’ suggests that deploying the same conventional pads across different intelligence stations was as much against the rules as was the ‘re-use’ over time of pads by a single pair of stations. Alternatively, it could mean that London-Moscow and Stockholm-Moscow both used the same reference-book in their emergency systems. In any case, this ‘re-use’ evidently occurred in 1940, well before the much publicized year of 1943 described in the VENONA histories as the time when the first infraction occurred. Andrew provides no guidance for his readers.

[ii] “GCHQ persuaded the Swedes to relinquish their neutrality, and pass the material over for analysis. The discovery of the Swedish HASP material was one of the main reasons for Arthur’s [Arthur Martin’s] return to D1. He was one of the few officers inside MI5 with direct experience of VENONA, having worked intimately with it during the Fuchs and Maclean investigations.

            There were high hopes that HASP would transform VENONA by providing more intelligence about unknown cryptonyms and, just as important, by providing more groups for the codebook, which would, in turn, lead to further breaks in VENONA material already held.

The first point here is a reminder of Sweden’s neutrality – not just during World War II, but during the Cold War, when it was not a member of NATO. Like Portugal and Switzerland, Sweden had been abuzz with spies during World War II, and its proximity to the northern ports of German-occupied Poland and the Baltic States meant that Stockholm was well-positioned to supply information on German naval capabilities, repairs, etc. Hence the feverish wireless communications with Moscow. Moreover, that neutrality apparently endured, so that Sweden would not have been a natural sharer of decryption techniques with NATO members. Yet Sweden was not ‘neutral’ enough to be free of suspicions about Soviet intentions, and thus pursued its own program of trying to gather wireless intelligence.

In Venona, Nigel West relates how the Swedes collaborated with the more advanced, cryptanalytically speaking, Finns, who had provided the American with highly useful aids when they handed over the partially burned Petsamo codebooks that had been retrieved from the Soviet consulate in June 1941. And, no doubt, informal links were in place between the Swedes and the British, as Wright’s text suggests. West even indicates that the Finns managed to understand how the Soviets ‘built code-tables and relied on a very straightforward mathematical formula to encode emergency signals’, but it is not clear exactly how this happened, or whether the lessons learned applied to the GRU as well as to the NKVD.

Wright’s suggestion here, however, is that HASP was, in essence, different from traditional VENONA, although it is not immediately obvious in what manner. The implication is that HASP would share much with the VENONA traffic, such as the use of the same codebook (the reference by which otherwise meaningless sequences of numbers represented terms, functions, identities of persons, countries, institutions, etc., sometimes known as a nomenclator).  The studies of VENONA tell us that the different functions of Soviet commercial organisations and intelligence (Amtorg, NKVD, GRU, Naval GRU and Foreign Ministry) used different code-books, and thus breakthroughs in one area did not mean that other successes naturally followed. For instance, all departments referred to the Germans as ‘KOLBASNIKI’ (’SAUSAGE-DEALERS’), but in the NKVD book, that word could have been represented as, say, ‘1146’, and in that of the GRU, ‘9452’.

This system was all independent of one-time pads for further encryption. Yet, if Andrew’s description is correct, Wright’s concluding sentence in this extract makes more sense. If the Swedes had managed to make inroads into the GRU codebook from the analysis of their local messages, that experience would transfer directly to the British study of GRU traffic. The emphasis on ‘VENONA material already held’ is telling. Wright is starting to backtrack from his original characterisation.

[iii] Moreover, since powerful new computers were becoming available, it made sense to reopen the whole program (I was never convinced that the effort should have been dropped in the 1950s), and the pace gradually increased, with vigorous encouragement by Arthur, through the early 1960s.

            In fact, there were no great immediate discoveries in the HASP material which related to Britain. Most of the material consisted of routine reports from GRU offices of bomb damage in various parts of Britain, and estimates of British military capability. There were dozens of cryptonyms, some of whom were interesting, but long since dead. J. B. S. Haldane, for instance, who was working in the Admiralty’s submarine experimental station at Haslar, researching into deep diving techniques, was supplying details of the programs to the CPGB, who were passing it on to the GRU in London. Another spy identified in the traffic was the Honourable Owen Montagu, the son of Lord Swaythling (not to be confused with Euan Montagu, who organized the celebrated ‘Man Who Never Was’ deception operation during the war). He was a freelance journalist, and from the traffic it was clear that he was used by the Russians to collect political intelligence in the Labour Party, and to a lesser degree the CPGB.

Some of this is puzzling. Unfortunately, a detailed history of the evolutionary progress of the VENONA decrypts is not possible, based solely on the selection of documents released. As West writes in his Introduction: “Whereas the American policy appears to have provided a measure of protection to the living, being those suspected Soviet sources who were never positively identified or confronted with the allegations, their British partners seem to have adopted political embarrassment as their principal criterion for eliminating sensitive names. The only other deliberate excision in the declassified documents is the consistent removal throughout of all references to the first date of circulation. Each VENONA text is marked with the last, and therefore most recent, distribution, but it is impossible to determine precisely when the first break in a particular message was achieved, or to chart the subsequent program of the cryptographers.”

Overall, West’s statement is accurate, although some decrypts (such as those on BARON) do reveal a series of release dates, and others have had the issuance date deleted. Unfortunately, many of the critical items related to HASP, such as the discovery of the X Group, have no release dates at all, so it is impossible to determine how much of the messages had been decrypted before the contribution of the HASP codewords – and code-book. Wright’s seemingly authoritative view is that the project was suspended in the early 1950s, and then reactivated at the end of the decade, but the redacted (or concealed) data on the issuance of new decrypts does allow us to create only a very partial evolution of texts through time.

All this information described by Wright appeared as original VENONA material when described by West in Venona (pp 62-63), and it can clearly be traced by studying the on-line archive. So why does Wright revert to ‘the HASP material which related to Britain’? He appears to be going back to his initial position, that HASP consisted of traffic intercepted by the Swedes. That might have reinforced the idea that HASP was a motley set of messages that included local Stockholm-Moscow GRU/KGB traffic as well as interceptions of wireless messages between London and Moscow – and maybe more. Yet that scenario continues to look unlikely. And if these reports were ‘routine’, presumably familiar through VENONA messages already deciphered, why did Wright not say so?

J B S Haldane

Furthermore, he introduces Haldane and Montagu as if their appearance were no surprise, and not scandalous. Haldane’s cryptonym was INTELLECTUAL and Montague’s NOBILITY: when did Wright learn that? The appearance of these cryptonyms would not have been ‘routine’ if this was the first occurrence, and their identities were not known. In fact, it would have been a stunning discovery to learn that one of Britain’s most respected scientists was a named spy. The fact that they were dead was irrelevant – except when it came to GCHQ’s heightened protectiveness about references to hallowed public figures, and maybe to their survivors. Wright’s manner here is astonishingly casual.

It does not help that Nigel West (pp 75-81) presents the discoveries about Group X and Haldane as standard VENONA traffic without mentioning any contribution from HASP. He confidently identifies INTELLIGENTSIA as J. B. S. Haldane, and NOBILITY as the Honourable Ivor Montagu. After all, West’s understanding of HASP was that it concerned Stockholm-Moscow traffic: he writes that the arrival of HASP allowed the project to ‘be put back into gear’, but does not explain how that happened. West provides a lot of useful and fascinating information about Haldane’s background and activities, but (for example) sheds no light on how the decryption of the codeword INTELLECTUAL took place.

Christopher Andrew, however, is more explicit on this portion of the traffic, although he, too, still does not mention HASP, and the description of it as ‘new’ VENONA is misleading and unfortunate. “The main discovery from this new VENONA source was the existence of a wartime GRU agent network in Britain codenamed the ‘X Group’, which was active by, if not before, 1940. The identity of the leader of the Group, or at least its chief contact with the GRU London residency, codenamed INTELLIGENTSIA, was revealed in a decrypted telegram to Moscow on 25 July 1940 from his case officer as one of the CPGB’s wealthiest and most aristocratic members . . .” Thereafter, Andrew rather surprisingly goes on to identify INTELLIGENTSIA as Ivor Montagu, instead of ‘Montagu’s friend’, J. B. S. Haldane. In an endnote (p 926, No 81), Andrew states that ‘West misidentifies NOBILITY as Ivor Montagu and INTELLIGENTSIA as Haldane’, but provides no argument for this. Certainly the meaning of the two cryptonyms would appear to suit West’s interpretation better.

In 2012, Nigel West amplified his previous analysis in the Historical Dictionary of Signals Intelligence, where he added further detail: “. . .  this unexpected windfall consisted of 390 partially deciphered messages, exchanged with Moscow between December 1940 and April 1446 [sic!]. The FRA had succeeded, as early as 1947, in reading a few messages, and between 1957 and 1959, some 53 texts were broken out. Information identifying individual Soviet spies had then been passed to the Allmänna Säkerhetstjänsten (General Security Service), which conducted investigations that effectively neutralized them without compromising the source.”

Apart from the vagueness of such terms as ‘broken out’ (does that mean complete decryption?), such level of detail is impressive, and authoritative-sounding, and West piled on the authenticity by naming eighty NKVD cryptonyms that provided ‘depth’ to the VENONA cryptanalytical process, including names that would carry import for the Washington and London operations, such as DORA, EDWARD, FROST, GROMOV, and  LEAF. West then listed an even longer array of GRU codenames, nearly all unfamiliar to me. But he did explain that, in August 1942, Lennart Katz ‘a source run by a contact working under diplomatic cover named Scheptkov, was arrested’, and provided further leads. It sounds as if West had access to insider information (Venona provides an Acknowledgement to ‘Stefan Burgland and some others who prefer to remain anonymous’), and that those arrested may have been able to provide insights on the ciphers and codes used. Moscow, however, appeared not to have worked out what was going on, and how so many of its spies had been detected.

[iv] The extraordinary thing about the GRU traffic was the comparison with the KGB traffic four years later. The GRU officers in 1940 and 1941 were clearly of low caliber, demoralized and running around like headless chickens in the wake of Stalin’s purges of the 1930s. By 1945 they had given way to a new breed of professional Russian intelligence officers like Krotov. The entire agent-running procedure was clearly highly-skilled and pragmatic. Great care was being taken to protect agents for their long-term use. Where there seemed poor discipline in the GRU procedures, by 1945 the traffic showed that control was exerted from Moscow Center, and comparisons between KGB and Ambassadorial channels demonstrated quite clearly the importance the KGB had inside the Russian State. This, in a sense, was the most enduring legacy of the VENONA break – the glimpse it gave us of the vast KGB machine, with networks all across the West, ready for the Cold War as the West prepared for peace.”

This section is mostly irrelevant to the quest. It is difficult to discern what Wright is talking about when he does not provide samples of the messages. The KGB’s operation in London was (we have been told by several experts) suspended for nearly all of 1940, so the GRU was the only game in town. And these ‘headless chickens’ did manage to recruit Klaus Fuchs, and manage a ring of useful scientists, such as Haldane. What he may have been alluding to was the somewhat casual way that information was supplied in telegrams, but that would have been more a case of insufficiently well trained officers, cipher clerks, and wireless operators – which were evidently in short supply at the beginning of the war –  rather than the quality of those who recruited and handled British agents. Kremer’s struggles with setting up the alternative wireless link may be an example of what Wright was thinking of.

Pp 238-239      “Lastly there was the VENONA material – by far the most reliable intelligence of all on past penetration of Western security. After Arthur [Martin] left I took over the VENONA program, and commissioned yet another full-scale review of the material to see if new leads could be gathered. This was to lead to the first D-3 generated case, ironically a French rather than a British one. The HASP GRU material, dating from 1940 and 1941, contained a lot of information about Soviet penetration of the various émigré and nationalist movements who made their headquarters in London during the first years of the war. The Russians, for instance, had a prime source in the heart of the Free Czechoslovakian Intelligence Service, which ran its own networks in German-occupied Eastern Europe via couriers. The Soviet source had the cryptonym Baron, and was probably the Czech politician Sedlecek [sic], who later played a prominent role in the Lucy Ring in Switzerland.”

Wright’s restricting of the ‘HASP GRU material’ to 1940 and 1941 is provocative, not solely because he now seems to be classifying HASP material as GRU messages collected locally. Is the temporal phrase ‘dating from 1940 and 1941’ merely adding chronology for the full scope of the material, or is it a qualifying phrase that subdefines a portion of it? The parenthesis, separated by commas, suggests to me the former, namely ‘the only GRU material that can properly be classified as HASP is that of 1940 and 1941’. Yet we have no way of knowing what GRU material had been attacked, and partially decrypted, before 1960, apart from various clues provided by the ‘experts’.

The rubric around the published VENONA messages is disappointingly vague. Yet there appears to be some discernible order behind the numbering scheme. In my analysis of the traffic between March 1940 and August 1941 (the last date in that year for which a message from London to Moscow has been published), I counted 137 L-to-M messages, with the first numbered (by the GRU) as No. 120, and the last as No. 2311. Yet a countback to zero seemed to occur at the beginning of each year. The last listed in December 1940 is No. 1424, while the first listed for 1941, on January 16, is No. 83. Thus one might assume that well over 4,000 messages were sent by the London station in those two years.

The Moscow to London traffic is sparser, with only 18 messages listed. The last calendar entry present for 1940 is from September 21, numbered as 482, so it would appear that Moscow was not so active sending messages to London, although the record would suggest that the combination of RSS (Radio Security Service) and GC&CS was picking up far fewer inbound messages, both in aggregate and proportionately, than it was outbound. But that could also be explained by a far smaller proportion of inbound messages being (partially) decrypted, or even a larger amount being for some reason concealed.

These numbers correspond closely with what Andrew has written (see above), where he refers to 178 messages between the period March 1940 and March 1942. Yet the autumn/winter of 1941/42 was clearly a period where activity of some sort (number of transmissions, number of interceptions, number of partial decryptions, number of released decryptions!) declined rapidly, and this is such a controversial aspect of the whole business that I shall return to it after completing my analysis of Wright’s text.

As for the remainder of this passage, the information, again, is not breathtaking, but Wright, alongside his rather laid-back commentary on Sedlacek [sic], does suggest by his comments that GCHQ had decrypted nothing on the Czechoslovak agent before the HASP project came along. Sedlacek [BARON] was a familiar figure in the VENONA traffic (see West, pp 67-69), and he played a dangerous game spying for the Swiss, the Czechs, the Russians – and the British, who later supplied him with a passport under the name of Simpson so that he could enter Switzerland and contribute to the Lucy Ring. Again, Andrew differs in his analysis of BARON, quoting (page 926, Note 82) an unnamed MI5 officer as saying, in 1997, that no serious attempts had been made to identify him. Why anyone would expect an MI5 (or MI6) officer to be open and straightforward about such a controversial figure as Sedlacek (if indeed that was who he was) is puzzling. Andrew attempts to reinforce his argument by noting that the NSA regards BARON as unidentified, but interest in these local European matters is unsurprisingly muted on that side of the Atlantic.

BARON indeed figures prominently in these messages: he was potentially very useful to Moscow as he was clearly passing on, in the run-up to Barbarossa, information about German troop movements in Czechoslovakia, Poland and Hungary, gained via his contacts around Prague who were transmitting information to him via Woldingham. I write ‘potentially’ because, of course, Stalin ignored all intelligence about the German invasion as ‘provocation’.

P 374-375 [i] “There had recently been a small breakthrough in the existing traffic which had given cause for hope. Geoffrey Sudbury was working on part of the HASP material which had never been broken out. Advanced computer analysis revealed that this particular traffic was not genuine VENONA. It did not appear to have been enciphered using a one-time pad, and from the nonrandom distribution of the groups, Sudbury hazarded a guess that it had been enciphered using some kind of directory.

This, again, is distressingly vague. By alluding to ‘HASP material that had never been broken out’, Wright again gives the impression that HASP was a collection of London-to-Moscow (or Moscow-to-London) communications. Why would Sudbury work on native Swedish transmissions? Presumably, ‘genuine VENONA’ to Wright was traffic that had become decipherable because the Soviets, under pressure, disastrously re-used one of their one-time pads. Distributing fresh pads was an enormous task in war-time, so the London-Moscow GRU link may have resorted to a different system whereby page-numbers and word-numbers in a shared book were used for encipherment schemes. Such a mechanism was essential for any transmission activity by clandestine agents, where the problems of distribution and security with one-time pads would have been insuperable. Leo Marks composed easily memorable verses for use in the field by SOE agents: the GRU used statistical almanacs for in-house use.

On the surface, Wright’s description of Sudbury’s analysis would appear, however, to be reinforced by the few accounts of GRU espionage that we have. A classical description of the use of one-time pads has the original cleartext (the passage in native language) immediately processed by a portion of the one-time pad, normally the next page, which would then be destroyed. In many accounts of the Soviet system (e.g. James Gannon’s Stealing Secrets, Telling Lies), that was the only method. Yet some accounts indicate that the GRU used a different process of encipherment. Benson’s in-house history of the NSA informs us that Igor Gouzenko described the method during his interview by Frank Rowlett in October 1945, when he revealed the back-up system of using a shared reference book in place of classical one-time pads. (Oddly, in his CIA report, Cecil Phillips, who assisted Nigel West in his researches, elides over this aspect of Gouzenko’s contribution.) In Appendix A to his 1949 book, Handbook for Spies, Alexander Foote (the Briton who was trained by SONIA as a wireless operator for the GRU in Switzerland) explains how a keyword of six letters, ‘changed at intervals by the Centre’ (and thus presumably communicated in later messages) was first used to translate the letters of the alphabet into a set of apparently meaningless numbers. Further manipulation transformed the text into five-figure groups – not yet a very secure encipherment.

Then came the ‘one-time’ aspect of the GRU’s process – but it was not through the use of a ‘pad’. Messages were then further processed by a function known as ‘closing’. Foote explained that, after the first stage of encipherment, he had to ‘close’ the message ‘by re-enciphering it against the selected portion of the “code book”’. (This ‘code-book, or ‘dictionary’ is a different entity from the ‘codebook’ that contained numeric representations of common terms.) This was a mechanism whereby a passage in a book owned by both parties was referred to by page and line number in order to identify a sequence of characters to be used to encipher a text one stage further. Max Clausen used a similar technique when enciphering for Richard Sorge, another GRU agent, in Japan. Foote said that he used ‘a Swiss book of trade statistics’:  David Kahn writes that Clausen used the 1935 edition of the Statistiches Jahrbuch für das deutsche Reich. Thus, for the GRU, the one-time pad was not a miniature printed guide that could be easily destroyed, but an accessible but otherwise anonymous volume that could be used by both ends of the communication. (Christopher Andrew’s claim that the Stockholm residency and the London residency employed the same one-time pads is thus probably not true: they almost certainly used the same – or a similar – reference work, however.) Sudbury had indeed hit upon the truth, and a directory was at work. This is what must be meant by ‘not genuine VENONA’.

What should also be recorded on this topic is the claim that Richard V. Hall makes in his ineptly titled but engrossing study of Wright and the Spycatcher trial, A Spy’s Revenge, that Wright acted as a ghost writer on Handbook for Spies. Since Wright was still working at the Admiralty Research Station in 1949, and did not join MI5 until 1955, this claim should be viewed circumspectly. If true, Wright’s apparent unawareness, in 1970, of GRU enciphering techniques is even more inexcusable.

[ii] We began the search in the British Library, and eventually found a book of trade statistics from the 1930s which fitted.

At first glance, this represents an enormous leap of faith. From ‘some kind of directory’ to stumbling on a book of trade statistics, with the implication that many others had been tested and found wanting first? Can it really be believed? That that is how the process worked, and that cryptologists would stumble on the right book? They must surely have been able to exploit a message that described the volume to be used, or gained a tip from someone. Suddenly, Alexander Foote’s hint of a ‘Swiss book of trade statistics’ takes on new significance. Wright echoes Foote’s words almost completely. Foote had died in 1956 (somewhat mysteriously: I am sure that Moscow’s ‘Special Tasks’ team was after him), but was surely interviewed on these matters at length by MI5 and GCHQ before he died.

Thus the dominant reaction should be: why on earth were Sudbury and Wright not familiar with Foote’s publication? It seems quite possible that they arrived at this conclusion by other means – namely what the Petrovs told them, and how Vladimir’s overall cryptological skills and knowledge, and particularly Yevdokia’s experiences as a NKVD cipher-clerk in Stockholm, benefitted the FRA, and in turn helped GCHQ. Yevdokia had worked for the GRU in her first eighteen months with OGPU, so she may have had some insight into its coding techniques.

After their post-war assignment in Stockholm, Vladimir Petrov and his wife had arrived in Australia in 1951, and decided to defect in 1954. Nigel West writes that Evdokia ‘was debriefed by western intelligence personnel, among them MI5’s George Leggett, who travelled to Australia after the couple had been resettled on their chicken-farm . . .’ Yet what Evdokia told them has not been disclosed. Was she responsible for GRU coding and encipherment, as well as that of the NKVD/MGB/KGB? Almost certainly not, but if so, she might have been able to inform the Swedes of such items as the name of the code-book (dictionary) used, and they thus were able to make some progress on the texts they had stored before the British did anything. If she had no involvement with the GRU, she might have been able to indicate the type of research volume that was used, and repeated efforts by Sudbury on the few relevant books of trade statistics in the British Library must have eventually borne fruit. Wright’s claim becomes clearer. It looks, however, as if the Swedes kept their project to themselves until 1959, when, for some reason, an informal link must have been elevated to an official communication.

[iii]  Overnight a huge chunk of HASP traffic was broken. The GRU traffic was similar to much that we had already broken. But there was one set of messages which was invaluable. The messages were sent from the GRU resident Simon Kremer to Moscow Center, and described his meetings with the GRU spy runner, Sonia, alias Ruth Kuzchinski [sic].

This is very dramatic – ‘overnight’, but, again, Wright dissembles and confuses. If the traffic was suddenly ‘broken’, he suggests that ‘HASP’ was in the hands of GCHQ already, but in a poor state of decryption.  Now, HASP appears to mean ‘GRU traffic derived from both Stockholm and London’. But why next characterise it as ‘the GRU traffic’ – what else could it be? And what does ‘similar to’ mean? Were they the same messages, enciphered differently? Was there really nothing new in them worth recording? And his reference to ‘one set of messages’ is also ambiguous. He gives the impression that this was a new trove of London-Moscow traffic supplied by the Swedes, when we now know that that cannot be true.

Certainly, one meeting between Sonia and her handler is recorded in the VENONA transcripts, dated July 31, 1941. The full item appears as follows:

“From London to Moscow: No.2043, 31 July 1941

IRIS had meeting with SONIA on July 30. Sonia reported (15 groups unrecovered):

Salary for 7 months: 406

John:  195

?? from abroad:  116

Expenditure on apparatus (radio and microdots):  105

?? Expenditure:  55

She played [broadcast] on 26, 27, 28 and 29 July at 2400, 0100, 0200 hours  . . . but did not receive you. BRION

(Comments by translator: IRIS probably a woman, IRIS means either the flower, or a kind of toffee. Unlikely choice for covername. JOHN probably Leon BUERTON [sic] BRION probably SHVETSOV, Assistant Military Attaché.)”

Yet the handler here is not Kremer: IRIS is probably Leo Aptekar, a GRU officer registered as a chauffeur at the Embassy. The annotation here about BRION is wrong: BRION has been confidently identified in the Vassiliev Notebooks as Colonel Sklyarov, for whom Kremer worked. Wright (and the VENONA website) identify Kremer as the rezident, i.e. senior GRU officer in London, but that does not appear to be the case. In Venona (1999), Nigel West described Kremer as being Sklyarov’s secretary, but in his 2014 Historical Dictionary of British Intelligence, West declares that the position was a cover for his ‘residency’, citing Krivitsky’s warning about him from 1940. Gary Kern (the biographer of Krivitsky) reflects, however, on the fact that others claim that Sklyarov was the boss. My analysis suggest that Sklyarov may have been brought in because Kremer was struggling, and Kremer then probably reported to Sklyarov after the latter arrived in October 1940. After all, Kremer turned out to be an unsuccessful cut-out for Fuchs, a role he would have hardly attempted had he been head-of-station. This is Pincher’s conclusion, too.

Sandor Rado (DORA) & I. A. Sklyarov (BRION)

One of the irritating aspects of the Venona archive, as published, is that identification of codenames switches from page to page, and the identification of BRION is one such casualty, with the annotators not being able to make up their minds between Sklyarov and Shvetsov. Vladimir Lota, in his ‘Sekretny Front General’novo Shtaba’ (Moscow 2005), confirms that BRION was Sklyarov, and offers a photograph of the officer (see above). West selects one VENONA annotator’s analysis that the reporting officer was Shvetsov, but informs us that Shvetsov died in an air accident in 1942. (The source of this is not clear. The Petrovs record that the family of an unnamed London military attaché died in transit from Aberdeen to Stockholm in 1943, when the plane was shot down over Swedish territory by German aircraft, but suggest that the attaché himself was not on board. See Yuri and Evdokia Petrov’s Empire of Fear, p 165).

As for Kremer, Mike Rossiter, the author of a biography of Klaus Fuchs, writes that he returned to Moscow in 1941, while West indicates that he remained in London throughout the war. Thus it is quite possible that Kremer composed reports on Sklyarov’s behalf, although his role had hitherto been as a courier. It was he who met Fuchs in August 1941, and he was Fuchs’s courier until the latter found he could not work with him, whereupon Fuchs was handed over to Sonia in the late summer of 1942. Kremer was also handling members of the X Group, so it seems unlikely that, at the same time that Kremer was regularly meeting Fuchs, he would also be meeting Sonia frequently, and then writing up the reports for Moscow.

The VENONA London GRU Traffic archive informs us that Kremer [BARCh]  ‘was appointed in 1937 and is thought to have left sometime in 1946. The covername BARCh occurs as a LONDON addressee and signatory between 3rd March 1940 and XXth October 1940, after which it is superseded by the covername BRION.’ (This analysis relies on the surviving VENONA traffic only, of course.) BRION first appears as a signatory or addressee on October 11, 1940. Thus the HASP traffic might provide evidence that Kremer was still active, as courier or signatory, or both, or, alternatively, the VENONA records might throw doubt on Wright’s claims about HASP. All three officers (Kremer, Sklyarov, Shvetsov) were active in London on June 7, 1941, as they are all cited as donating part of their salaries to the Soviet government.

The bottom line on Wright’s observations is that we are faced with another paradox. Apart from the fact that no trace of the ‘set of messages’ exists (why not, if they were solved overnight?), the association of Kremer with Sonia is very flimsy. The instance above is the sole surviving message in the VENONA archive that mentions SONIA. Wright’s account would imply the following: Apparently out of frustration with the fact that her transmissions received no response from Moscow, Sonia managed to contact the Embassy, and to meet her handler within a day or so. Sklyarov reported this event. At some stage afterwards, she was transferred to Kremer, who, apart from handling Fuchs, now had occasion to meet Sonia several times, and to make reports that he signed and sent himself. Yet the official archive informs us that Kremer stopped signing messages himself before Sonia even arrived in the United Kingdom.

What is also noteworthy is that Wright makes no comment about Sonia’s ability to escape radio detection-finding at this stage. If Sonia, as Kremer had recorded, had been transmitting for four successive nights, and had not been detected by RSS, one might have expected him, as a senior MI5 officer, to have reflected, at least, on her success in remaining undetected. He appears, at this stage, not to subscribe to the Chapman Pincher theory that Roger Hollis was able to interfere in the process; neither does he show any awareness that the proximity of Sonia’s home near Kidlington Airport might have masked her transmissions – which would admittedly have been a remarkable insight for that time.

[iv] The Sonia connection had been dismissed throughout the 1960s as too tenuous to be relied upon. MI5 tended to believe the story that she came to Britain to escape Nazism and the war, and that she did not become active for Russian Intelligence until Klaus Fuchs volunteered his services in 1944.

Apart from an evasive non sequitur (the connection was held to be tenuous, but MI5 accepted that Sonia became active with Fuchs in 1944, a very solid interrelation), Wright enters dangerous territory here, with a vague and undated summary of what ‘MI5 tended to believe’. Fuchs, of course, volunteered his services in 1941, not 1944, and was in the United States throughout all of 1944. Yet Wright’s lapsus calami may reveal a deeper discomfort, in that he utterly misrepresents the pattern of events. According to the archives, after Alexander Foote had spilled the beans on Sonia’s activities in 1947, MI5 strongly suspected that Sonia had been working for the GRU in the UK. They were ready (or pretended to be so) to haul her in for questioning on the Fuchs case as early as February, 1950, before his trial was even over, apparently unaware that she had already fled the country! (The service probably connived at her speedy escape.) The Fuchs archive at Kew shows that in November 1950, and again in December, Fuchs, from prison, viewed photographs and recognized Sonia as his second contact. Wright was either hopelessly uninformed, or acting completely disingenuously.

[v] In particular GCHQ denied vehemently that Sonia could have been broadcasting her only radio messages from her home near Oxford during the period between 1941 and 1943.

            But Kremer’s messages utterly destroyed the established beliefs. They showed that Sonia had indeed been sent to the Oxford area by Russian Intelligence, and that during 1941 she was already running a string of agents. The traffic even contained the details of the payments she was making to these agents, as well as the time and durations of her own radio broadcasts. I thought bitterly of the way this new information might have influenced Hollis’ interrogation had we had the material in 1969.

The statement attributed to GCHQ, if it indeed was made – and Wright provides no reference – needs parsing very carefully. We should bear in mind that no GCHQ spokesperson may have uttered these words, or that, if someone did state something approximating their meaning, Wright may have misremembered them. He provides no reference, no date, no name for the speaker.

First of all, Sonia’s home. She had, in fact at least four residences during this period, but, if we restrict her domiciles to those where she lived after she became active, probably in June 1941, we have Kidlington (from that June) and Summertown (from August 1942). Summertown was in Oxford, not near it. Thus a reference to ‘her home’ expresses lack of familiarity with the facts. ‘Only radio messages’ is perplexing. Does it mean ‘only those radio messages sent from her home?’, thus suggesting she could have sent messages from elsewhere? Maybe, but perhaps it was just a clumsy insertion by Wright. The omniscience that lies behind the denial, however, expresses a confidence that cannot be borne out by the facts.

It would have been less controversial for GCHQ simply to make the claim that no unidentifiable illicit broadcasts had been detected, and that Sonia must therefore have been inactive. But it did not. It introduced a level of specificity that undermined its case. It suggested that Sonia might have been broadcasting, but not from her home. If Sonia had been using her set, and followed the practices of the most astute SOE agents in Europe (who never transmitted from the same location twice – quite a considerable feat when porting a heavy apparatus, and re-setting up the antenna), she would likewise have moved around.

For GCHQ to be able to deny that Sonia had been able to broadcast would mean that it had 100% confidence that RSS had been able to detect all illicit traffic originating in the area, and that, furthermore, they knew the co-ordinates of Sonia’s residence at that time. Thus the following steps would have had to be taken:

  1. All illicit or suspicious wireless broadcasts had been detected by RSS;
  2. All those that could not have been accounted for were investigated;
  3. Successful triangulation (direction-finding) of all such signals had taken place to localise the source;
  4. Mobile location-finding units had been sent out to investigate all transgressions;
  5. Such units found that all the illicit stations were still broadcasting (on the same wave-length and with the identical callsign, presumably);
  6. All the offending transmitters were detected, and none was found to be Sonia’s.

Apart from the fact that transmissions from Kidlington were masked by proximity to the airport, and Sonia’s traffic concealed to resemble military messages, GCHQ’s assertion requires an impossible set of circumstances: that, if and when Sonia had broadcast, the location of the transmitter would have been known immediately, and the RSS would have been able to conclude  that the signals could not be coming from Sonia’s residence. That was not possible. No country’s technology at that time allowed instant identification of the precise location of a transmission. Not even groundwave detection was reliable enough to ‘pin-point’ the source of a signal to the geography of a city, even. Reports and transcriptions of suspicious messages were mailed by Voluntary Interceptors to the RSS HQ at Arkley View, in Barnet! Sonia would have had to broadcast for over twenty-four hours in one session to be detected by a mobile unit operating at peak efficiency, supported by rapid decisions (which was never the case). GCHQ might have claimed to Wright that no illicit transmissions originated from the Oxford area, and therefore they could discount Sonia’s apparatus (if they knew she had one.) Yet, again, that would require RSS to have deployed radio direction-finding technology in order to locate the transmitter, and Sonia would surely have stopped broadcasting by then.

Thus GCHQ’s claim is logically null and void. If Sonia made only one transmission, from her home or anywhere else, she would never have been detected. If she made more than one, from the same location, she would (according to the RSS’s reported procedures) inevitably have been detected, interdicted, and prosecuted. And GCHQ’s claim that she made no transmissions is clearly false, as she did transmit from the semi-concealed site at Kidlington, which was apparently never picked up. (After the war, she broadcast from her next home, The Firs at Great Rollright, as Bob King of RSS has confirmed, but these events are strictly outside the scope of GCHQ’s claim here.)

Moreover, GCHQ (actually named Government Code & Cypher School, or GC&CS, during the war) was not responsible for intercepting illicit transmissions in 1941-1943: that was the responsibility of RSS, which reported to SIS. GCHQ took over RSS after the war. Institutional memory may be at fault.

Ironically, Wright then undermines the GCHQ statement as an unfounded ‘belief’, as if it were a vague hope rather than a matter of strict execution of policy. Thus, either Wright drills a large hole in the track-record of GCHQ’s inviolability, or his claims about Kremer’s reporting of ‘the times and durations’ of Sonia’s own broadcasts lack any substance – or a mixture of both, since, irrespective of Sonia, RSS may not have been perfect in its mission of pursuing all illicit broadcasts, as we know from its own files. And we also know from the VENONA transcripts that Sonia tried to contact Moscow on successive nights in July 1941, from Kidlington. Since RSS apparently did not detect any of these transmissions, GCHQ’s boasts of omniscience are flawed. Wright’s lack of expressed astonishment at the inefficiency of RSS is again a remarkable reaction. Moreover, why would Kremer report on such details of her transmissions, if she was successfully in touch with Moscow already? It was one thing to report her failure to get through, but these claims appear superfluous, even absurd.

How we treat this claim about Kremer’s reports on Sonia’s broadcasts depends very much on how reliable a witness one views Wright by now. As Denis Lenihan has pointed out to me, what Wright asserts contains so much fresh information that his claims should be taken seriously. On the other hand, I would say that the Kremer telegrams are simply too implausible to be considered as valuable evidence. That Sonia would have had a ‘string of agents’ by 1941, that they would need to be paid, that Kremer would consider it necessary to report to Moscow the details of recent successful transmissions she had made to Moscow, even the role of Kremer himself in meetings and handling Sonia, fail to pass the authenticity test with this particular analyst. West and Pincher apparently agree with me. West relegates the item to an endnote on page 70. Pincher ignores the whole matter: there is no mention of HASP in his Index to Treachery.

Lastly, we have to deal with the final claims. It would be very unlikely for a wireless message, sent to Moscow in 1941, to provide the information that Russian intelligence had specifically sent Sonia to the Oxford area, although that might be a reasonable conclusion for Wright to make. In addition, the claim that Sonia had rapidly acquired a ‘string’ of agents, and was seeking expenses for payments that she was making to these mercenaries, is very improbable. Where and how she acquired them is not stated, but any contact who might have been providing information to Sonia informally would have probably jumped with alarm if Sonia had suggested that he or she should be paid for such indiscretions. Even Sonia herself, in her memoir, stated that the informants she nurtured provided her with confidential information out of principle, not for payment.

Yet the most awkward part of this testimony is the declaration that MI5 did not have this evidence in 1969, when (so Wright claims) it might have helped with a more successful interrogation of Hollis. Wright explicitly indicates that the discovery occurred in 1970, or later. The critical discoveries that were made in the decryption of reference book-based random numbers for the process of ‘closing’ were revealed, however, in the 1960s. The VENONA records show that GCHQ tried to censor a series of the Moscow-Stockholm GRU traffic for the Version 5 release of the decrypts, and that the Swedes had to restore the excised passages in Version 6. I have studied all these messages: a few appear to have no relevance to British affairs at all, but several do specifically relate to the use of commonly owned books (knigi), and even identify the titles of the volumes. All these messages have an issue date in the mid-1960s.

We thus come to the conclusion that GCHQ and MI5 had four opportunities to learn of the use of a common book to be used by agents and clandestine embassy wireless when it was too dangerous to try to deploy conventional one-time pads: Gouzenko’s revelations in 1945; Foote’s disclosures in his memoir of 1949; the descriptions gained from questioning the Petrovs in 1954/55; and the experiences of the Swedish FRA when they handed over their decrypts in 1960. Practically all the final decryption work on GRU London-Moscow messages that was possible was completed during the 1960s, yet Wright tries to pass off the breakthrough by Sudbury, and the serendipity location of the directory in the British Library, as occurring in the 1970s.

[vi] Once this was known I felt more sure than ever that Elli did exist, and that he was run by Sonia from Oxford, and that the secret of his identity lay in her transmissions, which inexplicably had been lost all those years before. The only hope was to travel the world and search for any sign that her traffic had been taken elsewhere.

Over the four years from 1972 to 1976 I traveled 370,000 kilometers searching for new VENONA and Sonia’s transmissions. In France, SDECE told me they had no material, even though Marcel told me he was sure they had taken it. Presumably one of the Sapphire agents had long since destroyed it. In Germany they professed total ignorance. It was the same in Italy. Spain refused to entertain the request until we handed back Gibraltar. I spent months toiling around telegraph offices in Canada searching for traces of the telex links out there. But there was nothing. In Washington, extensive searches also drew a blank. It was heart-breaking to know that what I wanted had once existed, had once been filed and stored, but had somehow slipped through our fingers.”

This, again, is a very controversial statement. Wright refers to ‘Sonia’s transmissions, which inexplicably had been lost all those years before’. Yet mentions of Sonia’s transmissions have never surfaced until now: the HASP exercise concerned the GRU’s alluding to such messages. Wright has given no indication that any of Sonia’s transmissions had been intercepted, and he even cites GCHQ as saying she could not have operated her wireless set undetected. So, if they never existed, they never could have been lost. Moreover, the records of Kremer’s supposed transmission(s) have also been lost. Wright may have wished that he had them in time to interrogate Hollis, but he cannot even present them after 1970, when it was too late!

Thus an astounding aspect of Wright’s testimony is his apparent lack of curiosity in determining what happened to the missing messages. He does not investigate what policy might have led to these last sets of decrypted traffic to be buried or destroyed. Surely his named colleague Sudbury and his fellow-cryptologists must have kept some copies of these vital messages, or at least have some recall as to what happened to them? Yet Wright does not undertake a search domestically first, or invoke his associates’ help in establishing the truth, and hunting the transcripts down. He ventures no opinion on the fact of their possibly being destroyed, but simply looks overseas.

Maybe there was a glimpse of hope that other countries might provide further VENONA nuggets, but, since we now know that the Stockholm operation concerned local traffic only, the quest seems very futile. And why ‘telex offices’? Why Wright expected further evidence of Sonia’s transmissions to come to light in telegraph offices around the world is astonishing. In the United Kingdom, Sonia’s messages were illicit, and subject to surveillance, with Voluntary Interceptors dispersed around the country to pick up the ground-wave from suspicious transmissions. If, by any chance, her messages were noticed anywhere else, amongst all the other radio noise, it would have been remarkable for any institution, public or private, to have dwelled upon them long enough to transcribe and store them. And if GCHQ (RSS) was never able to detect them, why on earth would Wright expect some foreign entity to be able to do so?

In addition, the question was not whether ELLI existed or not, but who ELLI was, and how significant a player he or she had been, and when he or she had been active. If this is the piece that clinches the argument for the case that Hollis was ELLI, it stands on very unsolid ground. Exactly what the link was between Sonia’s ability to maintain a string of agents and the existence of ELLI is not made clear by Wright. Did Wright really believe that he would have been able successfully to confront Hollis with the transcripts of Sonia’s messages to Moscow, and challenge him on the grounds that he had been able to prevent superior officers in MI5, RSS and GCHQ from performing their jobs?

It all echoes the laborious claims made by Chapman Pincher that the only way that Sonia could have hoodwinked MI5, RSS and GCHQ so that they all turned a blind eye to her shenanigans was through the existence of an intriguer in the middle ranks of MI5 who was so devious that he could entice his colleagues to ignore the basic tenets of their mission. Presumably it was ELLI who, instead of warning Sonia that it might be dangerous for her to persist in her illicit transmissions from one single geographic location, somehow convinced RSS that its procedures could be put in abeyance, and the signals ignored, and, moreover, that corporate memory allowed this oversight to become enshrined in official statements of policy within GCHQ after the war.

The Remaining Questions

Two crucial questions arise out of all this analysis:

  1. What happened to the missing messages?
  • Why did Wright mangle the story so much?

So much evidence conspires to inform us that what has been released to the archive of London-Moscow GRU traffic is only a small fraction of what was actually transmitted. The period of intensity is July 1940 to August 1941, followed by scattered fragments into early 1942, and a vast gulf until the end of the war, in 1945. The sequential telegram numbers tell us that less than 2% of the messages in 1940 and 1941 have been published. We have no idea how busy the communication link was during the next three years. We must therefore consider two separate sub-questions: i) given the ‘overnight breakthrough’ described by Wright, why were more messages in the 1940-1941 period not decrypted?, and ii) why was there a drought from the winter of 1941-1942 onwards?

The first sub-question cannot be answered by external analysis, as we do not know whether all messages were intercepted, which of these succumbed to even partial decryption, and which then remained classified because of issues of sensitivity or confidentiality. I do point out, however, that the official US VENONA website informs us that GCHQ did not hand over to the USA 159 of the GRU messages (i.e. close to the number I highlighted earlier) until 1996 – after the general disclosure of the VENONA project, indicating a high measure of discomfort about the disclosures (such as the Group X information).

What is also significant is that, having been passed decrypts from the Swedish authorities, GCHQ actually removed sections of the translated text before passing them on (in Version 5) to the Americans, with the result that the Swedes had to restore (in Version 6) the excisions GCHQ had made. Thus many messages in the VENONA archive include the puzzling rubric in their headings: “A more complete version of British Government-excised messages previously released in fifth VENONA release on 1 Oct 1996.” These revelations would seem to prove the case that the Swedes had made partial decryptions of their local GRU traffic, that they send these translations alongside the original messages, to GCHQ. It does not explain why GCHQ thought it was its business to edit them before passing them on to the NSA, especially if they also passed back their treatments to the Swedes at the same time.  A close analysis of all the relevant changes in Version 5 and Version 6 would be desirable. As I have indicated earlier, many of them have to do with the disclosures about shared reference volumes.

The Drought of 1942-1944

The second sub-question lays itself open to deeper inspection, because of the availability of other sources. On the matter of the missing messages, we need to judge:

  1. Did they not exist?
  2. Did they exist, but were never intercepted?
  3. Were they intercepted, but never stored?
  4. Were they stored, but subsequently lost?
  5. Were they discovered, but not decrypted (even partially)?
  6. Were they decrypted, but then not released?

The first issue is especially fascinating, partly because of Alexander Foote’s experience (or, at least, how he reported it). In October 1941, the Germans were at the gates of Moscow, and the vast majority of Moscow’s government apparatus was moved to Kuibyshev (now Samara), over a thousand kilometres to the east. In his testimony to MI5 in 1947, Foote told his interviewers that, working out of Switzerland, he lost contact with his controllers in Moscow in the middle of October, and, a few days later, even cabled Brigitte (Sonia’s sister) in London to determine what had happened. He then claimed that contact was not restored until March 1942, when he resumed his broadcasts. (This is all in Handbook for Spies, as well.)

Yet the existence of this forced hiatus is belied on at least two fronts. The TICOM (Target Intelligence Committee) archive indicates that Foote reported regularly during those winter months. Moreover, his boss, Alexander Radó (DORA) was using either Foote or another operator to communicate regularly with Moscow, as his memoir Codename Dora describes, with frequent messages about German troop movements. Radó echoes Foote’s story about the interruption, but states that it was on October 29 that he sent a desperate message to Moscow Centre. Contact was resumed at the end of November or the beginning of December, and all dated messages from October (the texts of which appear in Radó’s book) were re-transmitted. A telling detail indicates that Foote indeed was the chief wireless operator at this time: a TICOM interception shows that he reported on the source LOUISE from Berlin on December 3, and a related message listed by Radó of December 9 similarly reported on LUISE’s intelligence from Berlin. It could well be that Foote’s claim about radio silence was inserted by his ghost-writer at MI5, Courtenay Young – but why?

Radó’s telegrams are confirmed by Lota, who transcribes several of Radó’s messages from this period, and even includes photographs of a few from 1942. A satisfying match can be made between a telegram received on November 27, 1941 (Lota’s Document No. 37, on page 353), and Radó’s original message created on October 27 (p 76 of Codename Dora), confirming the delay before ‘Moscow’ returned to the air, and, incidentally, discrediting Foote’s account. Thus one might have expected a similar interruption to have occurred in London. Ivan Maisky, the Soviet Ambassador, tells us otherwise, however. Molotov remained in Moscow, and informed Maisky by telegram on October 17th that ‘most of the government departments and the diplomatic corps’ had left for Kuibyshev. This date, and the fact of the almost total evacuation of the Soviet government, are confirmed by other memoirs, such as Tokaev’s and those of the Petrovs. Maisky does not tell exactly when communications were re-established, but hints it was after only a few days, and he was then able to resume full contact. Thus he would have been able to pass on to the GRU officers inside his embassy what was happening, and they would not have made futile attempts to contact their bosses. Maybe, after a month, however, the watchers got tired of waiting for something to happen, and dropped their guard?

Then there is the ‘government policy’ theory. In Defending the Realm (p 376), Christopher Andrew, following up his comments about British government approval of Soviet use on ‘set frequencies’ (see above), writes: ”These radio messages were initially intercepted and recorded in the hope that they could eventually be decrypted, but interception (save for that of GRU traffic, which continued until April 1942) ceased in August 1941 because of the need to concentrate resources on the production of ULTRA intelligence based on the decryption of Enigma and other high-grade enemy ciphers. Interception of Soviet traffic did not resume until June 1945.”

This must be partially true. Yet Andrew shows a remarkable disdain for the facts in his endnote to this section, where he adds: “Since the intermittent Soviet reuse of one-time pads, the basis of the VENONA breakthrough, did not begin until several months after the German invasion of the Soviet Union in June 1941, the messages intercepted and recorded up to August 1941 proved of little post-war value to GCHQ.” Au contraire, maestro! There was practically nothing that was useful that occurred after August 1941, as Andrew himself records a few pages later, when he describes the disclosure of Haldane and the X Group, from July 1940. Moreover, Andrew does not explain why interception of GRU traffic continued for so long, or what happened to the messages stored. The VENONA GRU files show only two messages from 1942, both fragments, from January 19 (London to Moscow) and April 25 (Moscow to London).

Whether resources had to deployed elsewhere is a dubious assertion, too. Much has been made of the famous Footnote supplied by Professor Hinsley, on page 199 of Volume 1 of British Intelligence in the Second World War, where he wrote that ‘all work on Russian codes and cyphers was stopped from 22 June 1941’, variously attributed to Churchill himself or the Y Board.  The Foreign Office had promptly followed up the Y Board’s edict by forbidding MI5 to bug the Soviet Embassy, or to attempt to plant spies inside the premises, but was apparently more relaxed about the activities of MI6 and GC&CS, which nominally reported to the Foreign Office. While it may have taken a while for the policy statement to seep through, we should note that the edict said nothing about stopping the interception and storing of messages.

Robert Benson’s in-house history of the NSA (of which a key chapter is available on the Web) contains far more direct quotations from British authorities, such as Tiltman, Dill, Marychurch and Menzies, than can be found (as far as I know) from British histories. It reinforces the message that interception of Soviet traffic fairly rapidly tailed off towards the end of 1942, and that, during 1943 and 1944 any messages that had been stored were actually destroyed, to the later chagrin of intelligence officers. But that was what the alliance with the Soviet Union meant: a severe diminution in attempts to exploit Soviet intelligence, and that pattern was echoed in the USA. Since, at that time, no progress had been made on deciphering Russian traffic, it may have made little difference. One might also point out that, unless RSS intercepted all traffic, and inspected it, they would not know which was GRU and which was not, which makes Andrew’s already puzzling claim about the extension for GRU until April 1942 even more problematic, unless RSS knew that the secondary clandestine line was for GRU traffic only. Moreover, Andrew does not present Hinsley’s argument as a reason for the cessation.

‘HASP’ Annotation to Soviet Messages Detected in 1942

Certainly the Soviet Embassy was watched, and traffic was being monitored closely in March and April 1942. As I write, I have in front of me (see photograph above) the page from the RSS file HW 34/23, which shows a set of daily messages intercepted from March 16 to April 16, with callsigns, that changed each day, also listed. Very provocatively, the word ‘HASP’ has been written in opposite the April 7 entry, in what appears to be an annotation of May 1, 1973, and on the following page appears ‘from Maisky to Cadogan April 1942’, as if Maisky had perhaps had to explain himself to the Permanent Under-Secretary at the Foreign Office. (One cannot be certain that the annotation ‘HASP’ refers exclusively to the April 7 entry, or whether its serves as a general descriptor. If the latter, it would appear that, in 1973, the observer recognized this set of traffic, coming from the back-up GRU transmitter, as generic HASP material, but it does not explain how he or she reached that conclusion.) Other sheets suggest the surveillance went on into 1943. Yet all the evidence seems to point to the fact that, because of the signals being received from the Y Board and the Foreign Office, and the volumes of Nazi traffic to inspect, traffic from the clandestine line was either ignored, or simply piled up unused, and was discarded. Moreover, it was remarkably late for Wright (or whoever was the annotator) to be making, in 1973, a link between the HASP material of 1959 and the RSS files of 1943.

Nevertheless, a completely new project to monitor Soviet traffic was started at the beginning of 1943. After Commander Denniston had been replaced by Travis as the head of GC&CS in January 1942, he moved to London to set up a team that would begin to inspect and attempt to decipher Soviet diplomatic messages. This became known as the ISCOT project, after its key contributor Bernard Scott (né Schultz), and it led to the discovery of a rich set of ‘Comintern’ messages between the Soviet Union and its satellite guerrilla operations, after Stalin had supposedly closed down that organisation. Denniston was also involved in direction-finding the illicit traffic of 1942 to the Soviet Embassy. Thus, even if GRU/NKVD messages classified later as VENONA were ignored, it could hardly have been because of scarcity of resources. In addition, Andrew never explains why interception suddenly picked up successfully again in June 1945, and why RSS/GCHQ had no trouble finding the frequencies and call-signs used by the GRU.

A tantalising aspect of this whole investigation is the lack of overlap between published records of the GRU, and interceptions stored as part of the VENONA program. Verifiable records taken from Soviet archives are very thin on the ground, and we should be very wary of claims that are made of privileged access. Lota’s book (mentioned above) is a valuable source, containing multiple texts, and even photographs. It concentrates very much on military matters, especially concerning the movements of Nazi forces in the Soviet Union, and thus does not touch the early aspirations of the ENORMOZ (atomic weapons research) project. The familiar name of Sklyarov (BRION) appears quite frequently, but the first example of his telegrams is dated September 23, 1941 (Document No. 25). The VENONA sample of intercepted GRU messages from London (visible at https://www.wilsoncenter.org/sites/default/files/media/documents/article/Venona-London-GRU.pdf ) shows regular communications from BRION up to August 28, 1941, followed by a sprinkling of fragments up to March 1942, and then a long hiatus until 1945. Lota’s coverage thus overlaps in time, but I can see no messages that appear in both accounts.

Lastly, I must include the maybe very significant possibility that the rival channel set up in the London Embassy was not taken seriously enough. The official VENONA USA website offers (in ‘The Venona Story’) a very provocative paragraph, which runs as follows:

“Hundreds of GRU New York messages remain unsolved. The loss to history in the record of the GRU in Washington is particularly noticed. Of the several thousand Washington messages from 1941 to 1945, only about fifty were decrypted, in spite of the best efforts of the United States and the United Kingdom. Unlike the New York GRU messages, where translations concern espionage, these few Washington translations deal with routine military attaché matters (such as overt visits to U.S. defense factories). However, a separate Washington GRU cryptographic system, which was never read, presumably carried GRU espionage traffic.”

One might ask: ‘How did they know about this “separate Washington GRU cryptographic system’”?’ And what does ‘never read’ mean? That it was not intercepted? How did they know it was GRU if they never ‘read’ it? If it had been sent via cable, it would have been accessible, like all the other messages. Are the USA authorities referring to a clandestine wireless system, perhaps? And, if so, why did they not close it down? The reason these questions are relevant is that we have ample evidence that the GRU in London did attempt to set up a clandestine wireless system, and, after considerable teething problems, were apparently successful. (Vladimir Petrov confirms that such an arrangement happened in Stockholm, as well.) As I suggested earlier, it is possible that the RSS had worked out that the clandestine channel was for the GRU only. The intense USA focus of the VENONA website, and the various books that have been published in the US, mean that this project has not received the attention it deserves.

A closer inspection of the London-Moscow GRU traffic reveals the evolution of the project. The documents in this file are unfortunately not in chronological order, but a careful review suggests that the first reference is in a report dated July 17, 1940, from London to Moscow, where it is evident that a transmitter/receiver had been received in the diplomatic bag, but that the instructions for its assembly and deployment were deficient. London has to ask Moscow for the measurements for the aerial for MUSE’s apparatus. BARCh (Kremer) had decided to install the set in the lodgings of the military attaché, as he considered it was not safe in the Embassy, where the NKVD was ever-watchful. (“The only ones to fear are the NEIGBOURS’ people, who are in so many places here that it is hard to escape their notice.” This remark would tend to contradict the well-publicised notion that the NKVD staff had all been recalled to Moscow during 1940.) A few days later, however, it appears that Kremer has been ordered to change his mind, and install the radio-set in the Embassy, and is making rather feeble excuses about the lack of progress. On July 26, Kremer complains that the receiver works on 100 volts, which means it would be burned out by the 200-volt current in the embassy, and a transformer did not work. On August 13, they are back in the attaché’s house, where alternating current is available, and MUSE plans to try again, as a telegram of August 27shows. Kremer requests a schedule for the following months.

On August 30, 1940, reference is overtly made to the ‘London GRU emergency system’. The operator MUSE had been heard clearly, on schedule.   Yet problems in communication begin to occur in September, and the Director begins to show impatience, reporting again on September 18 that MUSE’s message was not received in full. Maybe it was Kremer’s struggles that prompted the transfer of Sklyarov from New York. Kremer tries to get his act together. In a message of October 3, he remarks that Sklyarov’s arrival is impending. In the same message he reports that MUSE has had a successful communication with Moscow at last, and that she will be trying again on October 7. Yet it was not a proper two-way conversation. On October 10, 1940, one of the few messages from Moscow shows the Director informing Kremer of further problems receiving messages on the illicit line, with nothing received since September 18. The Director has to remind him of the correct wavelength, crystal, callsign, and time.

It takes Sklyarov himself to report on November 25 that MUSE is now ready to begin regular communication, and that is the last we hear of the link for a while. Presumably it worked satisfactorily. Yet a very significant message on July 31, 1941 indicates a hitch, and that MUSE has had to test communications again. Sklyarov asked Moscow how well they had received her. The reason that this could be so important is the fact that the only report on SONIA that appears in the extracts was transmitted the very same day, suggesting perhaps that the back-up system (for highly confidential espionage traffic) was not working. Similarly, the only message from this period referencing Klaus Fuchs is of a short time later, on August 10. It would seem, therefore, that Sklyarov had to resort to the diplomatic channel to pass on critical information. Nearly all of the messages in the intervening period (November 1940-July 1941) concern more routine military matters (as Wright reported), so the absence of any other information on SONIA, both beforehand and afterwards, could mean either that there were no reports, or that they were sent on the clandestine channel.

It was probably this traffic which excited RSS so much in the spring of 1942, when they tracked unauthorised wireless signals emanating daily from the Soviet Embassy, signals that displayed an unusual pattern of call signs. As I described above, Alexander Cadogan in the Foreign Office seems to have approached Ambassador Maisky about them, but may have received a brush-off. Yet why only one of these messages was annotated with ‘HASP’ is puzzling. It is as if the messages had been intercepted and stored, and one of them had been (partially) decrypted through the assistance of the HASP code-book. But, in that case, why only one? And where is it? Was it the missing message from Kremer claimed by Peter Wright to show SONIA’s recruitment of her nest of spies?

Moreover, one final crucial paradox remains, concerning the two rare messages I identified a few paragraphs earlier. In the 1940-1941 GRU traffic can be found only one message referring to SONIA (3/NBF/T1764 of July 31, 1941: transcribed above), and only one to Klaus Fuchs (3/PPDT/101 of August 10, 1941). The singularity is startling. In their book, Venona; Decoding Soviet Espionage in America, John Earl Haynes and Harvey Klehr offer (on p 439) a footnote on the Fuchs message, which describes Fuchs’s meeting with Kremer. Part of the note runs as follows: “This message is from a period antedating the Soviet duplication of one-time pads. Its decryption was made possible because the London GRU station in 1941 ran out of one-time pads and used its emergency back-up cipher system based on a standard statistical table to generate the additive key. British cryptanalysts working with the Venona Project recognized it as a nonstandard and vulnerable cipher and solved it, but not until well after Fuchs’s arrest.”

I found this analysis disappointingly vague. Apart from the unlikelihood of the GRU’s suddenly running out of one-time pads, the note did not indicate for how long the back-up system had to run, and how the problem of distributing new pads was resolved. I took a look at West again. On page 26, he writes: “The clerk [Gouzenko] also described the GRU’s emergency cipher system, and although this was considered at the time to have potential, it was never found to have been used apart from the 1940-41 London traffic, when the GRU apparently ran out of OTPs.” This was even more opaque. It threw the traffic for two whole years into the ‘back-up system’ bin, when a cursory inspection of the files indicates that the primary system was working well until Moscow and London started discussing the problem. Yet it rather wearily echoed the text that appears in The Venona Story, namely that ‘  . . . several messages deal with cipher matters — in 1940 to 1941, the London GRU used a so-called Emergency System, a variation of the basic VENONA cryptosystems. London GRU messages merit very close attention.’  Indeed.

I managed to contact Dr. Haynes by email, and asked him whether he could shed any light on the source of the footnote. He promptly responded, reminding me that two messages in the GRU trove from this period referred to the OTP problem, citing telegrams No. 410, of August 30, 1940, and No. 1036, of September 19, 1940. Yet Haynes and Klehr had cited 1941 in their note! These two messages were transmitted about a year before the phenomenon of the Fuchs and Sonia messages! How could an OTP problem remain unaddressed that long? Was the implication that the back-up system (using the reference book OTP on the diplomatic channel, as the new GRU wireless link was not yet working) was used for the next twelve months? How should this information be interpreted? I tactfully raised these questions with Dr. Haynes, but, even after conferring with Louis Benson, he has not been able to shed any light on the confusion over the expiration of the one-time pads, and the use of the back-up system, although Benson did offer the important information that he thought the British had ‘identified the standard statistical  manual used to generate the additive keys’. But no date was given.

The sequence of events between April 1940 and March 1942, the period that encapsulates the most frequent of the London GRU traffic, is so confused that a proper assessment must be deferred for another time. The primary problem is that both London and Moscow refer, in messages presumably transmitted using the standard diplomatic channel, exploiting conventional one-time pads, of the imminent exhaustion of such tools. In that process, they ask or encourage the immediate use of the back-up system. Yet it is not clear that all successive messages use that back-up system, as later messages make the same appeal. It might be that the pads were in fact re-used as early as 1940. One enticing message (1036, of September 19, 1940) talks about ‘the pad used having been finally destroyed’, as if it should have been properly destroyed earlier, but was in desperation, perhaps, employed again, against all the rules.

In any case, a possible scenario could run as follows. Coincident with the GRU’s plan to move Sonia to Britain, to create a new espionage network, it decided to establish a clandestine wireless channel to handle her potential traffic. The task was entrusted to Kremer, but he struggled with getting the apparatus to work, and Sklyarov was transferred from New York to take charge. The conventional connection was used until November 1940, when the clandestine line was made to work, at about the time Sonia prepared to leave Switzerland. It was thereafter used successfully, until an interruption at the end of July 1941 caused Sklyarov to use the standard diplomatic channel for a critical message about Sonia – the only one to have survived in VENONA. RSS appears to have noticed messages on the clandestine link, but, if it did indeed intercept them and store them, no trace has survived. It is probable that no messages on that line were ever decrypted (apart from fragments at the end of 1941, and the two 1942 messages identified earlier). If other messages concerning Sonia were picked up and analysed from the standard link, GCHQ and MI5 must have decided to conceal them. (I have outlined this hypothesis to Dr. Haynes.)

Why did Wright mangle the story so much?

This close inspection of Wright’s account in Spycatcher shows a glorious muddle of misunderstood technology and implausible explanations. So why did he publish such an incoherent account of what happened? I present three alternative explanations:

  1. Wright simply did not understand what had been going on.
  2. Wright understood perfectly what had been going on, but wished to distort the facts.
  3. Wright had forgotten exactly what had been going on.

Number 1 is highly unlikely. He had been recruited as an expert with scientific training, and had showed knowledge of audio-electronic techniques to the extent that he uncovered Soviet bugs on embassy premises. He must have understood the principles of wireless communication, and the practical implications of intercepting both cable and wireless traffic. Number 2 does not make sense, as the mistakes that appear in his narrative tend to undermine any case he wanted to make about the identity of ELLI and the pointers towards SONIA. The sentence I cited above (in Cable or Wireless) is so manifestly absurd that it should immediately have alerted any knowledgeable critic to the fact that something was awry. If Wright had wanted to place a false trail, or was on a mission, he would have ensured that he appeared as a reliable expert on the main issues, but inserted subtle twists in the subordinate texts – in the manner in which Chapman Pincher operated. Wright definitely wanted to incriminate Hollis, but overall did not think he was distorting the truth, even if he was part of the ‘conspiracy’ to obfuscate what happened in the VENONA project. If he did embroider his account with the inclusion of an improbable and unverifiable message, he surely did not think it would be considered important, or that he would be found out.

Regrettably, one must conclude that, by the time Wright came to put his memoir together, he was approaching his dotage. Even though he was only seventy-one years old in 1987, his health was not good: he had high blood-pressure, shingles, and diabetes. In his account of the events, The Spycatcher Trial, Malcom Turnbull repeatedly draws attention to Wright’s failing health and faulty memory, pointing out that, as early as 1980 (when Wright was only sixty-four) he was too frail to travel from Australia to the United Kingdom by himself. Wright did not remember clearly how everything happened, how the intelligence services were organized, what the processes behind VENONA were, or exactly what HASP consisted of. His book was effectively ghost-written by Paul Greengrass, who clearly did not understand exactly what he was told by Wright, and, by the time it came for Wright to check the text, he was probably simply too impatient in wanting to see the book published, and consequently did not go over carefully everything that Greengrass had written. He was not concerned about the details: he wanted to get back at MI5 over its mistreatment of him on the pension business, he needed the royalties, and he was focused on getting the message on Hollis out.

I believe that it is entirely possible that, in his summoning up the telegram from Kremer that reported on Sonia’s network and payments, Wright was recalling the July 31, 1941 message that I reproduced in full above. It does mention agents and payments, but was sent not by Kremer, but by Sklyarov (BRION), mistakenly identified as Shvetsov in the annotations. We should not accept Wright’s account simply because, at one time, he had been an expert and a reliable witness. In addition, later reports suggest that there was an untrustworthy, almost devious, dimension to Wright’s behaviour. In his book on the trial, Malcom Turnbull expressed surprise at Wright’s ‘too uncritical worship’ of his mentor, Lord Rothschild. In his 2014 memoir, Dangerous to Know, Chapman Pincher asserted that Rothschild and his wife Tess loathed Wright, and he implied that Wright had exerted some kind of blackmail over the pair by threatening to include a chapter in Spycatcher that described Tess’s ‘long relationship with Anthony Blunt’.

As I indicated earlier, Chapman Pincher does not use his sometime accomplice Wright’s ‘evidence’ in his comprehensive presentation of the case against Hollis. Given that Pincher clutched at every straw he could find, and was always willing to present testimony from anonymous but ‘authoritative’ sources, this omission is somewhat startling. All Pincher states on Sonia’s recruitment of agents (beyond Fuchs and Norwood) runs as follows: “There is also new evidence that she and Len may have recruited and serviced a further fellow German communist – an atomic scientist working at the Clarendon Laboratory in Oxford, whose wife Sonia had met socially.” (p 198 of Treachery) Pincher also acknowledges that members of her family were informants for her, but dismisses Sonia’s claims about finding and recruiting ‘minor agents’ as possibly being a ‘GRU legendary cover’ (p 259). What this ‘new evidence’ consisted of is not explained, and the first statement has a very hypothetical ring about it. The conclusion, however, must be that Pincher did not trust Wright’s account of the breakthrough telegram.

Conclusions

Apart from the fact that ‘Spycatcher’ caught no spies, Wright was an unreliable witness. As D. Cameron Watt observed about the case: “A moderately careful reading of Wright’s book, let alone any checking of such statements he makes that can be checked, reveals, as most serious reviews of the book in the American press have shown, that Mr. Wright’s command of the facts, let alone his claims to universal knowledge, are such as to cast the gravest doubts on his credibility where his assertions cannot be cross-checked.”  He completely misrepresented the structure of the VENONA project, and the material it used. He was likewise confused about the elements of the HASP program, and what the Swedes brought to the game. He magnified an illusory message, unlikely in its authorship, improbable in its content, and dubious in its objective, in order to promulgate a claim about Sonia that has no basis in any other facts, and to provide ammunition for a flimsy case that ELLI was Roger Hollis, the incrimination of whom he blatantly stated was his goal in publishing the book. In his muddled argument, he committed much damage to the other aspects of his case. At the time of the Spycatcher trial, even though he was only 71 years old, he was portrayed by Richard Hall and Malcolm Turnbull as an old, sick man, with a reputation for mendacity. He received the news of the outcome of the trial while in hospital.

The VENONA files, which should provide the archival evidence for his investigation, are in a mess. The USA website is very US-centric, it is scattered with spelling mistakes, chronologically misplaced items, contradictory and incorrect annotations about identities, misrepresentations of English place-names, and wayward references that could be cleaned up by recent scholarship. The British GRU traffic has been broken out, but it is out of sequence. An intense analysis of the pan-European communications could shed some strong light on a host of new relationships. A comprehensive index needs to be built, so that scholars could be more productive in bringing their expertise to bear.

HASP was a project that exploited GRU traffic between Stockholm and Moscow, which had been partially decrypted by the Swedes. It succeeded because of the policy that the GRU deployed, for the operations of clandestine and emergency services, and those of agents under their control, of using a common reference-book as a one-time pad. The Petrovs’ experience in Moscow and Stockholm contributed substantially to identifying the volume used. Thus dramatic improvements in decrypting certain London-Moscow traffic were made. Yet fresh work can be undertaken. The considerations of HASP, and other published material (e.g. Vassiliev), need to be incorporated into the British VENONA story (of which there is no ‘authorised’ publication at all, and nothing fresh since Nigel West’s book of 2009) and cross-referenced. An analysis of the excisions that the British Government is stated to have made between the Version 5 and Version 6 releases should be undertaken. In other words, it constitutes a major opportunity for GCHQ in the year that its authorised history appears. It needs a professional cryptanalyst to work on the source messages, and the evolution of the decipherment.

As I have written before, an authorised history of wartime and post-war interception services remains to be written. To begin with, the function crossed multiple organisations – not just all the intelligence services, but the War Office, the armed forces, the Post Office, even the Metropolitan Police. The Radio Security Service (RSS), of interest primarily to MI5, was never owned by the Security Service (despite Nigel West’s continued claims to the contrary), and was managed by a section of SIS from May 1941 until the end of the war, when GCHQ took control of it. Yet Keith Jeffery, in his authorised history of SIS, treated RSS (and GCHQ, which also reported to SIS during the war) as step-children. It will be interesting to see whether the coming history of GCHQ (Behind the Enigma, The Authorised History of Britain’s Secret Cyber Intelligence Agency, by John Ferris, due in November of this year), when covering the wartime years, treats RSS as an essential part of GC&CS (as it was then).

I believe that this bulletin provides an accurate account of the phenomenon of HASP, but a similar modern exercise needs to be performed against VENONA itself. After I post this report, I intend to draw the attention of the GCHQ Press Office to it. I ask all readers who would like to see some effort expended on clearing up this significant episode in British Intelligence History to contact the Press Office at pressoffice@gchq.gov.uk themselves, and thus reinforce my message.

(I regret that this research has been conducted without detailed access to the several files on VENONA at the National Archives, which have not been digitized. My previous superficial scans of the information did not indicate to me that the matters I have discussed were covered by the archival material at all. If any reader has found information in them that either clarifies, expands or confounds what I have written, please contact me. I also want to express my gratitude to Professor Glees, and to Denis Lenihan, for comments and suggestions they made concerning an earlier version of this article. Denis has continued to provide, right up to the completion of this report, very useful insights from the material he has analysed. I alone am responsible for the opinions expressed here, and any errors that may appear in the text.)

Major Sources:

Spycatcher, by Peter Wright

Venona, by Nigel West

GCHQ, by Richard Aldrich

The Code Breakers, by David Kahn

Stealing Secrets, Telling Lies, by James Gannon

Handbook for Spies, by Alexander Foote

The Code Book, by Simon Singh

Battle of Wits, by Stephen Budiansky

Stealing Secrets, Telling Lies, by James Gannon

Historical Dictionary of Signals Intelligence, by Nigel West

Sekretnyi Front General’nogo Shtaba’, by Vladimir Lota

Venona: Soviet Espionage and the American Response 1939-1957, ed. Robert Louis Benson & Michael Warner

Defend(ing) the Realm, by Christopher Andrew

The Haunted Wood, by Allan Weinstein & Alexander Vassiliev

Venona: Decoding Soviet Espionage in America, by John Earl Haynes & Harvey Klehr

The Venona Secrets: The Definitive Exposé of Soviet Espionage in America, by Herbert Romerstein & Eric Breindel

The Secrets of the Service, by Anthony Glees

The Secret History of MI6: 1909-1949, by Keith Jeffery

Empire of Fear, by Vladimir and Evdokia Petrov

Between Silk and Cyanide, by Leo Marks

Codes, Ciphers & Other Cryptic & Clandestine Communications, by Fred B. Wrixon

British Intelligence in the Second World War, Volume 1, by F. H. Hinsley and others

The Venona Story, by Robert L. Benson

MI6 and the Machinery of Spying, by Philip H. J. Davies

The Petrov Affair, by Robert Manne

A Spy’s Revenge, by Richard V. Hall

The Spycatcher Affair, by Malcom Turnbull

Treachery, by Chapman Pincher

Dangerous to Know, by Chapman Pincher

Peter Wright and the ‘Spycatcher’ Case, by D. Cameron Watt, in Political Quarterly, Volume 59, Issue 2, April 1988

The National Archives

https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/venona-soviet-espionage-and-the-american-response-1939-1957/preface.htm

https://www.wilsoncenter.org/sites/default/files/media/documents/article/Venona-London-GRU.pdf

https://www.nsa.gov/Portals/70/documents/news-features/declassified-documents/venona/dated/1945/16jul_cipher_text_seaman.pdf

https://nsarchive2.gwu.edu//NSAEBB/NSAEBB278/01.PDF

https://vault.fbi.gov/Venona/Venona%20Part%201%20of%201/view

https://digitalarchive.wilsoncenter.org/collection/86/vassiliev-notebooks

This month’s new Commonplace entries can be found here.

2 Comments

Filed under Espionage/Intelligence, General History, Literature/Academia, Politics, Technology

The Mystery of the Undetected Radios (Part 6)

Virginia Hall of SOE & OSS operating, with Edmond Lebrat pedalling a generator, in July 1944 (by Jeff Bass)

The previous chapter of this story concluded by describing the state of events in the autumn of 1942. It had been a difficult year for the Allies, but the tide of the war had begun to turn in their favour. The five-month battle of Stalingrad, which represented the Soviet Union’s critical effort to repel the Wehrmacht, began in October, and the USA’s arsenal was beginning to have an effect in the rest of the world. Nazi Germany accordingly intensified its efforts to eliminate subversive threats, and by this time had rounded up the sections of the Red Orchestra operating on German soil, executing many of its members in December. The Allied landings in North Africa (November) prompted Germany to occupy Vichy France, which removed a safer base of operations for espionage and sabotage work originating in Britain. Meanwhile, Churchill had ended his opposition to the Overlord invasion plan in a deal over sharing of atomic research and technology with the USA. Colonel Bevan had thus been appointed to reinvigorate the important London Controlling Section, responsible for strategic military deception, in August 1942, and serious plans for the invasion of Europe were underway. Yet Bevan had a large amount of preparatory work to do, and circulated his draft deception plan for the broader theatre of war, Bodyguard, only at the beginning of October 1943. It was approved later that month, with refinements still being made in December. All domestic intelligence agencies would be affected by the objectives for the segment describing the European landings, named Fortitude.

This (penultimate?) chapter takes the story of wireless interception up to the end of 1943, and again concentrates on the territories occupied by the Nazis in Central Western Europe – the Low Countries and France, with a diversion into Switzerland, as well as the domestic scene in Great Britain. Roosevelt had founded the OSS (the Office of Strategic Services, roughly equivalent to MI6 and SOE) in June 1942, and thus Britain’s dominant role in European resistance began to fade. The rather haphazard approach to sabotage that had characterized SOE’s work up till then began to evolve into a more considered strategy to support the invasion. It was placed under closer military control in March 1943. The uncertain role of Britain’s Double-Cross agents received a much sharper focus in preparations for a campaign of disinformation to deceive the Germans about the location of the landings. The RSS started to concentrate more on the challenge of locating ‘stay-behind’ agents in Europe than on the detection of illicit domestic transmissions in the United Kingdom. Yet issues of post-war administrations began to surface and introduce new tensions: as the Red Army began to move West, Churchill and Eden started to have misgivings about the nature of some nationalist movements, SOE’s associations with communists, and Stalin’s intentions. Moreover, Roosevelt’s OSS was much more critical of Britain’s ‘imperialism’ than it was of Stalin’s ‘communist democracy’, which also affected the climate with the various governments-in-exile in London.

The Reality of German Direction- and Location-Finding

Whereas the missions of the various German interception services had previously been focused on the illogical basis of the political motivations of the offenders, in 1943 a split based on geography was initiated. The WNV/FU assumed control for Northern France, Belgium and South Holland, the Balkans, Italy, and part of the Eastern Front, while the Orpo (Ordnungspolizei) was given responsibility for Southern France, the rest of Holland, Norway, Germany and the rest of the Eastern Front. This may have led to differences in operational policy, and equipment used: little intelligence-sharing went on, however, because of political rivalries. In the previous chapter I had suggested that the scope and effectiveness of the German direction- and location-finding machine had been exaggerated by the Gestapo as a method of deterrence, and that, in reality, infiltrated wireless operators were betrayed more by shoddy practices and informers. I now examine this phenomenon in more detail.

A popular reference work on espionage (Dobson and Payne, 1997) describes the operation as follows:

“German direction-finding operations in France were centered on Gestapo headquarters in the Avenue Foch in Paris. Relays of 30 clerks monitoring up to 300 cathode-ray tubes kept up a continuous watch on every conceivable frequency between 10 kilocycles and 30 megacycles. When a new set opened up it showed at once as a luminous spot on one of the tubes. Alerted by telephone, large goniometric stations at Brest, Augsburg and Nuremburg started to take cross-bearings. Within 15 minutes they were able to establish a triangle with sides about 16 km (10 miles) across into which detector vans from a mobile regional base could be moved to pinpoint more precisely the area of transmission.

Typically, a mobile regional base would be equipped with two front-wheel-drive Citroen 11ight vans, each crewed by four civilians carrying machine guns, and two four-seater Mercedes-Benz convertibles with fake French licence plates. If the transmission had ended the vehicles would move to the intersection points of the triangle and wait in the hope that the unknown station would acknowledge a reply to its message. An acknowledgment of a mere three to four seconds would allow an experienced team to reduce the sides of the triangle to no more than 800 m (0.5 mile). If the transmission were longer, the operator would almost immediately be compromised.”

I see several problems with this account. First of all, it contains no dates, no sense of gradual establishment. I have not discovered any images of the CRT equipment claimed to be deployed. If a transmitting set were to be detected without high-powered interception stations working in harness first, it would have to be via ground-wave, which would be restricted to a distance of about ten miles. That limitation would not justify the huge expense required in the centre of Paris, since most illicit transmissions occurred in the provinces. In any case, the assumed illicit signals would have to be discriminated from all the other police, military and industrial activity going on at the same time. The number of personnel, vehicles and equipment to cover the whole of France would be astronomically high, and, especially at this advanced stage of the war, Germany did not have an available competent and dedicated labour force to deploy successfully in such a project. How many ‘mobile regional bases’ were there? It would have been a colossal waste of resources to deploy this infrastructure on the assumption that occasional illicit transmissions could be promptly identified and eliminated.

This dubious reference attempts to shed light on the process by means of an imaginative diagram:

German Direction-Finding

The text for this entry is echoed almost verbatim in Jean-Louis Perquin’s The Clandestine Radio Operators (2011), a work that boasts a serious bibliography and set of sources. Here a few additional details are supplied by the author. The German unit is identified as the Kurzwellenüberwachung [Short-Wave Observation], or KWU, with a codename for the operation of DONAR. (I cannot find any other reference to a such-named unit – a true hapax legomenon?)  “A total of one hundred and six men, seven mobile goniometers mounted either on trucks or on one of the service’s 35 cars was made available”. The author adds that protection was provided by the French Sureté Nationale. Yet the mechanisms are vague. “A control station equipped with over 300 (ultra-modern) receivers continuously monitored over thirty thousand frequencies  . . .”  The principle behind the scheme was that any unregistered frequency used was ‘highly likely to signal a covert radio-operator’. Then a telephone message was immediately sent to the three direction-finding centres in Brest, Augsburg and Nuremberg, which would quickly be able to determine an equilateral triangle of 20 kilometre sides in which the operator was transmitting. Thereafter, the trucks were sent in to the tip of the triangle, sometimes supported by a team of pedestrian monitors using sensitive magnetometers on their wrists. In that way, they would quickly identify the building where the transmission was occurring, and arrest the agent before he or she committed suicide.

The operation was claimed to be very efficient.  “This was the procedure used in 1943. If the clandestine transmitter was located in the same city as a mobile goniometer base, the location of the transmitter could be identified within a 200-metres radius in less than a quarter of an hour.” Further: “As an example, the German DF could be within sight of a transmitter half an hour after it sent its very first signal. It is likely that, by the spring of 1944, the Germans were using a fully automated, car-mounted DF system using a cathodic screen monitor.” The official historian of SOE, M. R. D. Foot, may be the originator of this particular histoire, writing, in 1984: “The Germans, like the British, kept a constant watch on every wireless wavelength, and it took only twenty or thirty minutes for a team of their armed direction-finders to get within a few yards of an operator who was fool enough to remain on the air so long. Relays of thirty clerks with cathode-ray tubes in the Gestapo’s headquarters in the Avenue Foch in Paris, for example, kept up a continuous watch on every conceivable frequency. When a new set opened up, it was bound to show up on a tube; the frequency could be read off at once. In a couple of minutes, alerted by telephone, direction-finders at Brest, Augsburg and Nuremberg were starting to take cross-bearings; within a quarter of an hour, detector vans would be closing in on the triangle a few miles across that the cross-bearings had indicated.”

It seems as if these accounts were also received by the RSS, which at the end of the war compiled a report on the Funkabwehr (available at HW 34/2 at the National Archives).  The writer lists the claims made by captured German officers, and ‘various sources’, illustrating them with such dramatic detail as: “Within a period of two minutes each new suspect signal was observed and reported by line to a large scale system of D/F networks which could obtain bearings with an error of less than half a degree and so plot the position of any station to an area within a radius variously estimated at from 4 to ten kilometres. This process required a further seven minutes, after which five further minutes were necessary to bring a very strong mobile unit organisation into action and for them to proceed by short-range D/F and shifting to locate the transmitter.” The report then casts serious doubts on the reliability of these statements, which appear to be the work of German propaganda, sent out by various media, in an attempt to discourage Allied wireless use.

The RSS report includes some details about mobile unit operations: that the 1942 Operation Donar in the Unoccupied Zone was largely ineffective, as few French-speaking persons took part, and it was very obvious; that a single mobile unit roamed around Southern France in 1943, ‘principally Marseilles and Lyons, until it settled in Lyons’ (which does not suggest dense coverage); that the communications between interception and the D/F stations in the OKW were poor, certainly not as good as the Orpo’s; that effectiveness was hindered by personnel transfer; that local and atmospheric conditions greatly hindered accurate readings; that many cases were recorded where the mobile units were totally unable to locate the groundwave. In certain cases, mostly in urban areas, a very focused operation could produce results, especially when the famous ‘guertel’ snifter (the Gürtel Kleinpeiler für Bodenwelle) was introduced in 1943, but, overall, location-finding was a very haphazard affair, and nothing like the streamlined operation that the authorities liked to represent.

There is no reliable evidence of the number or names of clandestine operators who were caught by this method. It should be concluded that there must be a large amount of propagandizing in this scenario, with no reliable source provided. As previous incidents have shown, there is no dependable way of identifying the physical source of a ‘new’ message stream over the ether unless something is known about the data sent – the callsign, for instance, which may have been revealed through torture or collaboration. Only when triangulation occurs could the rough proximity of the transmission zone be determined. And the operator would have to continue transmitting for an inordinate amount of time for the detectors still be able to sense him or her when they eventually turned up in their vans. Moreover, part of agent practice was to employ ‘watchers’ who would look out for the tell-tale features of the DF vehicles, and agents were taught to stay on the air for only a few minutes at a time before signing off and moving location.

Inside a Gestapo DF Truck

The whole process is belied by some of the autobiographical accounts that were published after the war. Jacques Doneux’s They Arrived By Moonlight is considered one of the most reliable descriptions of the life of a clandestine radio operator – this time in Belgium. He explains how he managed to evade the direction-finding vans, by transmitting at different times of the day, by varying the location, by staying on the air for no more than half an hour, and by using a protection team to warn him of approaching vans. Significantly, one of the statements he makes runs as follows (p 105): “We went to a place called La Hulpe which was a short way out of Brussels and fairly safe from direction-finding; this meant that we could have a good long sked with little fear of interruption.” This suggests that urban detection capabilities were based on ground-waves, and that the mechanisms for intercepting and trapping illicit broadcasters were much less sophisticated than has frequently been claimed. (I return to Doneux when discussing SOE later in this piece.) Another technique used with some success by the territorial guardians, however, was the deployment of radio-detecting planes. Doneux reports that ‘a Fieseler-Storch, flying low, often appeared about ten minutes after an operator had started to transmit’. This very visible and obvious mechanism clearly encouraged radio operators to be brief. The RSS Report on the Funkabwehr claims, however, that the Fiesler-Storch was equipped to operate where mobile units could not go, namely the Russian Front and the Balkans.

Perquin presents a more down-to-earth analysis at the end of his article, where he breaks down the record of SOE’s F Section. “For ten arrested radio operators, at least five fell victims to carelessness of breaches of basic security rules; another two arrest [sic] could have been avoided had the transmissions not been sent from cities where German DF teams had regional branches. Many radio operators like other members of resistance networks were compromised because of careless talk, gossip, indiscretion, police investigations or sheer bad luck in the form of a routine police check. On the other end, the fact that ten radio operators were captured should not hide the extraordinary usefulness and effectiveness of the remaining ninety if one is to mention only F section. ‘Kleber’, belonging to the French intelligence branch and not to the SOE, never had a single incident when it used its eight transmitters to send signals to Algiers from the immediate vicinity of Pau (SW France). By 1944, the average duration of a transmission was less than three minutes per frequency.”

In summary, the existence of location-finding teams is not in doubt, but they were certainly far fewer in number than claimed by some expansive reports. They may have picked up some random operators. Yet, rather than a comprehensive mechanism for picking up previously unknown operators, it is much more likely that the system was deployed to try to mop up remaining members of a network whose predecessors had already been betrayed by some source or behaviour, when the general neighbourhood in which they were working was already known. Promoting the mythology of a powerful and ruthless machine may however have acted as a useful deterrent for the Nazi security organs, and ascribing failure to it may have served to absolve leaders and remote directors of resistance groups of lapses in security procedures.

The Red Orchestra

A more reliable model for how the Gestapo worked is provided by the successful efforts to close down the section of the Red Orchestra that operated out of neutral Switzerland. As I explained in the previous episode, the units of the Red Orchestra in Germany and France had been largely mopped up by the end of 1942, primarily because of atrociously lax inattention to security procedures by the Communist agents. (The executions at Plötzensee carried on until December 1943.)

Developing an accurate account of the operation of the ‘Rote Drei’ (as the main three wireless operators in Switzerland, Foote, Radó and Bölli, were known) is notoriously difficult. The memoirs of Foote – which were ghosted – as well as those of Radó, are highly unreliable, and the source of much of the strategic intelligence, probably gained from Ultra decrypts, is still hotly contested. The authoritative-sounding analysis emanating from the CIA is also riddled with disinformation. For a refresher on the background, I refer readers to ‘Sonia’s Radio’, especially http://www.coldspur.com/sonias-radio-part-vii/.

German Intelligence had been intercepting the messages of the Soviet agents in Switzerland since November 1941, but apparently no headway had been made on decrypting them. Then, as the German network was being closed down, the volume of messages from across the border increased. According to V. E. Tarrant, in The Red Orchestra: “During the latter half of 1942 the German long-range radio monitoring stations in Dresden and Prague reported heavy radio traffic from three short-wave transmitters operating in neutral Switzerland. Through cross-bearings two were tracked to Geneva, close to the Franco-Swiss border, and the third to Lausanne on the northern shore of Lake Geneva.”

In January 1943, with the German network rounded up and executed, attention thus switched to group in Switzerland, and the pressure mounted for making sense of the transmissions, and determining how vital and accurate they were. OKW/Chi (Chiffrier Abteilung) was charged, on February 23, with attacking the messages, and, perhaps surprisingly, made swift progress, an achievement which suggest, perhaps, that some work had been undertaken in a more dilatory fashion before then. Tarrant again: “When the intercepts of these transmissions were sent to the radio traffic analysts in the Funkabwehr offices on the Matthaïkirchplatz they concluded that the cipher employed by the Swiss operators was of an identical format to the one-time pad that had been used by the Grand Chef’s [Trepper’s] pianists.” Tarrant suggests that the agent ‘Kent’, who was in the custody of the Gestapo, helped in the deciphering process. In any case, the CIA reported that Chi had gathered all the extant traffic by the end of March, and in a few days had discovered the main principle of the encryption technique. By April 22, sixteen messages had been broken.

The first reaction by German Intelligence was to conclude that the information was of the highest quality, and continued dissemination could seriously damage the war effort. Yet the organs found it very difficult to identify a Berlin-based source responsible for the information, or the medium by which the information could have been passing. (I shall not re-explain here the claim that ‘Lucy’, the enigmatic Rudolf Rössler, was in fact receiving his intelligence from the United Kingdom, itself deriving form Ultra decrypts.) Instead, they resolved to track down the suspects in Switzerland. Their location-finding techniques could identify the cities from which the transmissions were being made, but Switzerland was of course neutral territory.

Radó’s network took a fairly relaxed attitude towards security. The Swiss Government was reasonably tolerant of foreign intelligence activity, so long as it was not directed against Switzerland itself. The unit considered itself free from the observations and threats of the Gestapo, and was under enough pressure from Moscow Centre, in the latter’s persistent requests for identifying sources, and the torrents of questions that they presented to Radó and his team. Thus the Germans had to use a combination of traditional espionage and political pressure to help them track and close down the dangerous wireless trio.

In 1941 (or, according to some accounts, 1942), Walter Schellenberg had been appointed by Himmler to head Section VI, the RSHA’s foreign intelligence branch. Indeed, he had already had clandestine meetings with the Swiss intelligence chief, Roger Masson, in the summer and autumn of 1942, after Masson had heard rumours that the Germans were planning an invasion of the country. Yet Schellenberg’s intentions in setting up the meeting may have been to persuade Masson to cooperate in prosecuting the Rote Drei. Max Hastings, in The Secret War, informs us that Schellenberg told Masson then that Berlin had already decrypted two of the ring’s messages, and was seeking help. The threat of invasion, which was always a real threat to the Swiss, because of its German-speaking population, and Hitler’s designs on the ‘Südmark’, was a not-so-gentle incentive for Masson to ‘help with the RSHA’s inquiries’. The two met again, early in 1943. It appears that Germany had made serious demands that Switzerland maintain its neutrality, under threat of invasion, and Masson did indeed crumble, and deploy his native counter-intelligence experts to mop up the illicit wireless network.

The Gestapo had also tried inserting agents to subvert and betray the network, but these were mostly clumsy efforts that Alexander Foote was able to deflect. The mopping-up operation did not take long, however. In September 1943, the Swiss Bundespolizei (BUPO) began the operation to silence the transmitters. They used the traditional goniometric techniques to locate the equipment more accurately, starting with Geneva. Since the agents were not accustomed to moving premises, or having to restrict the length of their transmissions (Foote recorded being on the air for hours owing to the volume of work), there was no rush. Tarrant even reports that ‘it took a few weeks for Lt. Treyer’s direction-finder vans to pin-point the actual locations . . . ‘. That luxury would not have been available in the pressure-cooked environment of Belgium or France. The BUFO also used the famous method of turning off the power to houses individually in order to notice when transmission stopped. And the frailties of war-time romance took their effect, as well. Margrit Bölli, one of the wireless operators, took a lover, Peters, who was in fact a German agent and stole her cipher key. She ignored instructions, and moved to his apartment, where BUFO agents tracked her. The Hamels were arrested on the night of October 13/14, and just about a month after that Foote himself was arrested.

Radó escaped into hiding, and some abortive attempts to resuscitate the network were made, but they fell short – primarily because of funding. Ironically, BUFO tried to carry on a ‘Funkspiel’ (along the lines of what the Germans performed in the Netherlands) with the Soviets. Foote had owned a powerful wireless set, capable of reaching Moscow, obviously, but also the Americas, and Treyer, in possession of Radó’s code, initiated messages in German on Foote’s set, using that code. Yet, as David Dallin inform us in Soviet Espionage, ‘Foote’s previous messages, always in English, had usually been transmitted in his own code’. (The Soviets deployed techniques for alerting Moscow Centre of code-switches to be deployed in a following suite of messages.) The Soviets saw through the ruse very quickly.

Because of the sympathetic role that the spies had been playing in support of Switzerland’s resistance to Nazism, they were all treated relatively well. Yet an important source of intelligence was closed down. By then the Battle of Kursk (to the success of which the Lucy Ring had substantially contributed) was over, the Wehrmacht had been mortally damaged, and the war was as good as won. From the standpoint of illicit wireless interception, however, the story has multiple lessons. It reinforces the fact that remote direction-finding, across hundreds of miles, could be an effective tool in locating transmissions at the city-level. It shows that suborned and tortured agents, with knowledge of callsigns, schedules, ciphers and codes, could provide a much quicker breakthrough to decryption than laborious ‘blind’ brainwork. It stresses the importance of solid tradecraft and security techniques for agents to avoid successfully those in pursuit of them (although, in a small country like Switzerland, where their activities were suspected anyway, it would have been impossible for the Rote Drei to have held out for long). It emphasizes the role that simple security techniques could play in avoiding the successful ‘turning’ of networks. One other consequence of the operation was that Moscow stopped relying so much on the illicit transmissions of mainly ‘illegal’ agents, and switched its focus on using couriers and equipment in the Soviet Embassies to manage the traffic that their spies were still accumulating.

Exploits of SOE & SIS

I have earlier drawn attention to the renowned actions taken by General Gubbins in tightening up SOE security in 1943, and how they need to be questioned. Not only were these initiatives very late, the claims about their success are not really borne out by the evidence. Much has been written about the careful psychological screening of potential SOE agents, and their wireless operators, and even more has been written about their lengthy training in all manner of tradecraft as a foreign agent, from practice at parachute-jumping to secure methods of wireless transmission. Yet the experiences in France and the Low Countries, as recounted by M. R. D. Foot, tell of a parade of broken backs, legs and ankles resulting from clumsy parachute landings, of wireless sets that broke on impact, were lost, or simply did not work. It seems quite extraordinary that so much would be invested in preparatory training, only to be wasted in the minutes following the dropping of the parachutists. (Several of these highly trained wireless operators were killed in plane crashes.) SOE did not have the luxury of a rich labour pool from which to select the most suitable candidates, and the pressures on it to deliver were immense. Yet, despite the attention given to training, it was clearly deficient in many areas.

Moreover, procedures regarding wireless security were still inconsistently applied. Foot again: “It did not take long [sic] for Gubbins, as head of operations, to spot what was wrong, or for the signals training school at Thame Park to start to impress on operators – as Beaulieu explained to organisers – that mortal danger lay in trying to send long messages by wireless.” Yet the order that no transmission was to last more than five minutes did not go out until the winter of 1943-44. In September 1943 (when Gubbins replaced Hambro as head of SOE), more flexible and unpredictable ‘skeds’ (transmission schedules – a critical part of the software, since they had to take into consideration such factors as atmospheric disturbance) were introduced: irregular hours and switching of frequencies made detection more difficult.

What became necessary was a keen sense of how active the organs were in a particular area. Foot relates how, in May 1943, an agent named Beckers was able to stay at his set ‘for two hours without any trouble, and only once heard of a D/F car in the neighbourhood’. Another, Léon Bar, was quickly arrested after starting to address a backlog of messages, and tried to shoot his way out of trouble. He was tortured, and then killed, but it is not clear whether direction-finding or betrayal caused his demise. Wendelen escaped surveillance because he had an informer in the Vichy police, who warned him of all direction-finding efforts in the Indre département. Yolande Beekman successfully transmitted from same spot at the same hour on the same three days of the week for months on end during 1943 and 1944. It is somewhat shocking to read, however, that, in the summer of 1943, Wendelen returned to England, and had to make some fundamental suggestions for better tradecraft, such as water-proofing the containers, and requiring at least one look-out man during every schedule. Why did it take so long to learn and apply these lessons?

Yet some of the practices were not repeatable. Scheyven never transmitted from the same house twice, and remained undetected. Goffin learned from predecessors: “He kept his sets buried in large boxes in gardens; kept codes and crystals hidden in a different address; never carried his set himself. His case can stand for an example of how sensible SOE agents were able to benefit from the more foolish mistakes of others.” Agents on the run, with no variety of safe houses to choose from, could not afford such luxuries, and local residents became increasingly petrified at being found out by the Gestapo harbouring an illicit wireless operator. They knew the penalty. The operational pressures were imperfectly understood by the controllers in London.

Jacques Doneux’s memoir seems to be a more reliable guide to the psychological stress. He provocatively wrote that the locals, who had been working on subversive work much longer than any agent, were frequently dismissive of strict security procedures, preferring to rely on their own wits, and sense for danger. Doneux was certainly aware of detector vans, but always used a squad of look-out men, and paid solid attention to location and transmission-times. He was one who considered that Nazi claims of radio-detection efficiency were inflated (viz. his comment about moving to La Hulpe), but it did not take much for the transmission to be interrupted, and the carefully prepared sked ruined. Extra controls deployed by the Gestapo made walking around with a wireless transmitter even more perilous, so mobility caused fresh challenges.

Lastly must be considered the advances in equipment, especially when SOE set up its own workshop in 1942 on being freed from dependence upon SIS. One of its first breakthroughs was the S-Phone, which was designed to be worn on an agent’s chest, whereby he could make contact with an allied aircraft by voice, up to a distance of thirty miles, and to a height of 10,000 feet. This technology had the advantage of using UHF, and was not detectable by conventional D/F techniques owing to the highly focused antenna, and the low power consumption. The S-Phone was used primarily to guide arriving planes on drop areas or landing-sites, but was also used to convey brief instructions and information between the two parties. Articles published elsewhere indicate that the S-Phone had been deployed as early as 1941, which suggests that SOE was very early in its lifetime carrying on secret research while nominally still under the control of SIS.  William Mackenzie’s Secret History states, however, that ‘one of the very early uses of the S-Phone’ occurred only on July 22, 1943, when Lieutenant-Colonel Starr had been deprived of any regular wireless contact since November 1942, and had up till then had to rely on couriers through Switzerland and Spain. In any case, Gambier-Parry of Section VIII got to hear about the development.

The SOE’s S-Phone

Certainly, by 1943, smaller transmitters were being used for regular short-wave communication. Doneux refers to his carrying round his set under his overcoat. Foot describes the first innovations by F. W. Nicholls as follows: a Mark II in action by October 1942, 20lb in weight, which sent at 5 watts on 3-9 mc/s. Its successor, the B2 (technically, the 3 Mark II) was even more popular: it required 30 watts, and needed only two valves. It could transit between 3 and 16 mc/s, and could also receive. “None of the SOE’s sets suffered from a tiresome disadvantage of the paraset, which when switched to receive would upset any other wireless set in use for a hundred yards around: a severe brake on action in built-up areas where civilians were still allowed their own receiving sets.” The B2 weighed 32 lb., which sounds a bit bulky to be slipped under an overcoat, however. It was for longer ranges. Doneux may have been using the Mark III, which weighed only five and a half pounds, and fitted with its accessories into a tiny suitcase. Its 5-watt output could reach up to 500 miles.

In Western Europe, electric current was usually available, which meant that generating capabilities were seldom required. Matters were much tougher in other areas, such as Yugoslavia and Albania. During the same period, authors such as Deakin record the treks involved in lugging 48-lb transmitters and chargers driven by bicycle-type pedalling mechanisms across mountainous country. (A famous example with the OSS in France can be seen in the painting of Virginia Hall that I selected as the frontispiece to this article.) Mules were required to carry such a load, and in one memorable passage Deakin describes such a mule toppling into a crevasse, taking the equipment with him. For purposes nearer to home, successful miniaturization was slow to take hold: later in the war, when the Jedburgh teams were set up, a new small ‘Jedset’ was developed, but its fragility and size meant that it was frequently broken on landing. Not enough attention had been paid to insulating it from hard contact with the ground.

The SIS appeared to have greater success in 1943, although its mission of intelligence-gathering was subject to consistent interference from the sabotage objectives of SOE. With the invasion plans starting to be made, the demands made on SIS branches for information about German defences, installations, and troop movements, and research on potential landing-sites for the invasion, and the like, became more intense – and more immediate. Couriers were slow, which switched pressure to wireless communications.

The volume of information that was successfully passed back to London suggests that dozens, or even hundreds, of wireless operators managed to evade surveillance, and send their reports successfully across the airwaves. Keith Jeffery, in his authorised history of SIS, praises ‘Section VIII’s outstanding achievement in developing and refining radio transmitters and receivers’, which ‘made an indispensable contribution’. The author adds, however, that ‘at the sharp end it was up to individual men and women to operate the equipment in often very hazardous circumstances’. As an example, he cites the experiences of ‘Magpie’ in March 1943, who, pursuing loyally the strategy of trying to keep mobile, had to walk nine miles to his next safe house, during which journey the handle of the set broke twice, as it was not strong enough. Perhaps not such an outstanding job of design, after all. The answer was – more sets, a requirement to which Kenneth Cohen in London complied.

In Belgium, at the end of 1942, SIS also experimented with specialised ground-to-air communications, which allowed agents to communicate directly (and without the lengthy process of Morse codification) using the so-called ‘Ascension’ sets developed by Gambier-Parry’s team. (These were presumably similar to the technologies used by SOE. Indeed, an article in Cloak and Dagger suggest that the sets were an enhancement of the SOE invention: see https://www.docdroid.net/MEaQLK7/cloak-and-daggerair-enthusiast-2007-07-08-130.pdf ) Jeffery writes that ‘the Ascension sets were used with some success in Belgium and elsewhere, but the system was not very useful for long messages which still had to be smuggled out by courier across long and precarious land routes’.  That statement implies that long messages could not be trusted to conventional short-wave radio connections, because of the requirement to be on air for hours at a time, and the real or imagined threat of radio-detection techniques. Jeffery suggests soon afterwards that a lag of three or four months was occurring between information-gathering and receipt, and that the results were therefore valueless. By May 1943 even the courier supply lines had broken down.

Whether that problem was restricted to Belgium is not clear (remember the ‘elsewhere’). Certainly in France the networks were overall much more productive, despite a new set of challenges. A continual danger of a network’s having been suborned existed, but this threat was complemented by the onset of ideological disagreements between the various resistance groups, who, as the day of liberation became more real, each promoted their own view on what the political shape of the country should be after the war. For a while, the Gestapo appeared to use propaganda rather than competent feet on the ground, and anecdotal evidence suggests that the organisation was having trouble providing enough sharp and well-trained officers and men to control the noisy underworld. It frequently resorted to denouncers to make up for its deficiencies.

Yet, by the end of 1943, Madame Fourcade’s ‘Alliance’ organisation was almost completely destroyed – not by super-efficient surveillance techniques, but by Nazi infiltration of the groups. As Jeffery reports: “  . . . by the late autumn of 1943 most of the Alliance groups in north-west France and the Rhone valley had ceased to function”. Overall, communications out of France were considered to be inadequate, and the main channel for passing information was with a French diplomat in Madrid. Jeffery rather puzzlingly states that this person (named ‘Alibi’) ‘managed to establish wireless communications with networks in France’. This is one of the many enigmatic, vague and incomplete observations in the authorised history: no date is given, and the statement poses many questions. How were skeds set up? How many staff were on hand to receive messages, at what hours? And what did they do with them? Moreover, if a link could be made between networks in France and Madrid, how was it that the sources could not communicate with London directly?

The Evolution of the RSS

“James Johnston recalled in letters to me that he and his colleagues had intercepted messages from an illegal transmitter in the Oxford area, which he later believed to be Sonia’s, and had submitted them to MI6 or MI5. ‘Our logs recorded her traffic, but they were returned with the reference NFA [No Further Action] or NFU [No Further Use].’ According to Morton Evans, it was Hollis and Philby who decided that the logs should be returned to the RSS marked ‘NFA’ or ‘NFU”. This meant that the RSS was not required to send out its mobile detector vans. No such action was ever taken against Sonia during the whole duration of her illegal transmissions. ‘Her station continued to work, off and on,’ Johnston recalled. ‘It must be a mystery as to why she was not arrested.’’ (from Chapman Pincher’s Treachery, p 141)

This now famous passage by Chapman Pincher is extremely controversial, suggesting that the identity of Sonia was known to the authorities who monitored and instructed the interception plans of the squad of Voluntary Interceptors who scanned the airwaves. In this latest manifestation, it even identifies the senior RSS officer making the claim to Pincher, Kenneth Morton Evans, who, in a letter to Pincher, reportedly stated that gave ‘full details to Hollis in MI5 and Philby in MI6’, and implied that those two intelligence officers were unable to decrypt the messages.

That latter assertion is absurd, as neither Philby nor Hollis, had they indeed been passed the original texts, would have possessed the skills or authority to start trying to decrypt them. Yet it is the suggestion that the order to send out the mobile vans was withheld that is even more provocative. Earlier, Pincher had written: ‘The RSS had responsibility for locating any illicit transmitters. Detector vans with direction-finding equipment could be sent in the area to track down the precise position of a transmitter with police on hand to arrest the culprit. As a former operator James Johnston told me, ‘Our direction-finding equipment was so refined that we were able to locate any wayward transmitter’.”

Thus the objective observer, perhaps now familiar with the urgent security rules impressed upon SOE agents in Europe, has to accept the following scenario: Possibly illicit Soviet signals are detected emanating from the area of Oxford in the UK, perhaps identifiable by their callsigns. These are sent to the RSS discrimination unit, which studies them, and passes them to officers in MI5 and MI6. After these gentlemen get around to inspecting them (and perhaps attempting to decode them), it is their responsibility to say whether or not the transmitter should be located. If so, the vans are sent into action (perhaps a few days later), in the hope that the transmitter will still be obligingly cooperating by transmitting from the same place.

It is not the purpose of this analysis to determine whether the RSS was negligent over Sonia. This reader is convinced that she was left in place so that her transmissions could be surveilled. (Remember, on January 23, 1943, the Oxford police had visited Sonia’s residence, and reported to MI5 the discovery of a wireless set on the premises.) What needs to be established is how reliable is the testimony (if it truly exists) of Kenneth Morton Evans, a senior and capable wireless professional. From 1941 to 1945 he was the officer in charge at Arkley, the RSS facility that gathered and processed all the messages received by the Voluntary Interceptors. (In 1951, as an MI5 officer, he wrote a letter to the Guardian claiming that The National Association for Civil Liberties was a Communist front: see https://www.theguardian.com/uk/2002/jan/06/humanrights.world ) How was it that Morton Evans expected an illicit agent to hang around in the same location for several days? Was his understanding of the readiness and efficacy of the mobile vans accurate? Or was he also a party to the cover-up over surveillance of Sonia, contributing to the convenient story that Hollis had successfully protected her? And how did his account overall undermine the pretence that Nazi agents were able to work undetected in England for years?

The facts about the mobile detection apparatus are elusive. I have started to examine some of the historical records at the National Archives. [But not all: I am still waiting to receive photographs of many critical files, such as the WO 208/5099-5102 series. This section may thus require a later update. This analysis is based on WO/208/5096-5098, HW 34/18, HW 43/6, CAB 301//77, ADM 223/793 and FO 1093/484.]

Soon after the outbreak of war, Colonel Burke of MI8c (the forerunner of RSS) listed the equipment then in service, and made requests for expansion. His deposition ran as follows:

Direction Finding Stations      6 + 4

                                    Listening Stations                   4 + 2

                                    Mobile Vans                           10 + 14

                                    G.P.O. Detection Vans           88 (up to 200 available)

                                    Amateur Listening Posts        27

                                    Local D.F. systems for regional centres 1 + 16

                                    Transmitters for beacons        0 + 20

He added that ‘only one of the mobile vans is now fully equipped’, but that ‘the remaining vans should be ready in two to three weeks’. It is not clear what the distinction is between ‘mobile vans’ and ‘G,P.O. detection vans’. It could not be solely one of ownership: an earlier memorandum noted that the GPO provided the six fixed and ten mobile stations. It may have been one of designed function: a paper written in January 1941 records that ‘mobile vans (which were normally used to assist listeners in the detection and suppression of radio interference from industrial and domestic equipment) had been lent by PO to deal with the problem of detecting illicit radio beacons.’ Meanwhile the notion of ‘beacons’ (devices to assist arriving bombers to find their targets) had evolved to one of illicit transmissions. The Post Office was seen by military men as an unreliable, slow and bureaucratic organisation, unsuitable for holding responsibility for such critical tasks.

The official SIGINT history reinforces a rather casual approach to the use of mobile units: “Fixed interception stations would search the ether . . .  In the event of signals being intercepted, they would pass to the direction-finding stations the callsign, wavelength and text of the message. Supplementing this would be the widespread corps of voluntary interceptors whose function it would be to listen to the amateurs working in their area, observe their habits and report anything unusual. Mobile units were to perform the function of determining the exact location of the illicit transmitter. After the fixed D/F stations had located the general area of the transmitter, the mobile direction-finding units would proceed there, await further signals, obtain more accurate bearings and so narrow down the area of search.” And it indicates that, when the transmitter was located, the responsibility for what happened next would be MI5’s: the service might want to monitor it rather than close it down. (In that case, why sending out mobile vans, which might frighten the transgressor, and cause him to stop broadcasting, is not explained.)

But what happened to the expansion programme? It probably never occurred. As I have described before, by 1940 the interception mission of RSS was almost focused on overseas traffic. The History suggests a somewhat desultory approach could have been taken to what was then considered a non-problem. At some stage, a Mobile Units Group, under Major Elmes, centred in Barnet, controlled also the bases in Gateshead, Bristol and Gilnakirk, the establishment of which I described in the previous chapter. Fixed stations would then locate a general area of about 400 square miles. A report would be given to MI5, and the Mobile Unit organisation set in motion. At least three mobile vans were posted on the perimeter of this area, in contact with the Police Station in neighbourhood, a headquarters to which an MI5 officer would be attached. When the transmitter was heard, simultaneous bearings were taken by the Mobile Units and reported to HQ, where they were plotted on a map. The units then moved closer, and took fresh bearings ‘until definite action was possible on the part of the MI5 officer present’. But MI5 had no powers of arrest, and it is not clear what judgments the MI5 officer would be able to make on the spot in the event that a transmitter was caught red-handed. The narrative sounds like a good deal of wish-fulfilment, and post facto puffery for the historians.

Mobile vans definitely did exist, as Guy Liddell makes occasional reference to them in his Diaries. Yet, in 1943, as RSS started to consider the security needs for the invasion of Europe, it encountered fresh challenges. The History again informs us: “‘During this period RSS had accepted a further extension of its commitments without, however, affecting the vital features of its programme. This was the monitoring, by mobile units, of certain classes of signal made by our own stations, to prevent the inadvertent passage of information likely, if intercepted, to be of use to the enemy. the possibility of such leakage had been recognized and dealt with in the early days of the war by the cancellation of amateur transmitting licences and the impounding of transmitters, and the vetting of MI5 of firms requiring licenses for experimental or testing purposes. With GPO collaboration such action was easy to take, since licences were granted by that body. As the GPO did not necessarily license other Government departments however, it was found that there was a number of organisations using radio transmitters of which the Security Service had no official knowledge, as for example, experimental establishments of the Ministry of Supply, the Ministry of Aircraft Production, the Railways, the GPO stations themselves and Cable and Wireless stations. In addition the Police and the Fire Service possessed their own transmitters.” The organisation was under stress, and memoranda attest to the fact that its original mandate was being ignored.

A budgetary memorandum from 1941 indicated that capital expenditures for two Intercept Stations, at £25,000, and for Vehicles & Equipment, at £3,500, were requested. Annual Expenditures for P.O. Agency Services (D/F & Mobile Unit [sic!]) were estimated at £78,000. Yet, after some disturbing gaps in the record, the Estimates for RSS in the Budget Year of April 1942 to March 1943 include very little on mobile units, with Special Apparatus given as £10,000, and expenses of Mobile Unit Operations a mere £8000. This is not the high-powered, swift-moving organisation reportedly described to Chapman Pincher by Andrew Johnston and Kenneth Morton Evans, but a service apparently being rapidly wound down. (Were radio-detection vans perhaps later requisitioned and repurposed as transmitting vehicles to roam around issuing bogus signals of a phantom army? And an intriguing minute from D. I. Wilson of B1A in MI5, dated February 24, 1943, recommends that, if phantom armies were to be created, bogus wireless traffic needed to be realized as well, to support the false information to be passed on by the agents. Was Wilson perhaps the originator of one of the more spectacularly successful aspects of the whole OVERLORD operation?) Other memoranda written at this time indicate that the resources of RSS, including the reconstruction and repositioning of receiving stations at Hanslope, Cornwall and Forfarshire, and the installation of rhombic aerials, were being increasingly focused on mainland European needs.

Meanwhile RSS struggled to resolve its political problems in 1943, caused mostly by the over-secretive Cowgill, the highly-opinionated Trevor-Roper, the arrogant Gambier-Parry, and the manipulative Malcolm Frost. Frost left MI5 in November 1943 to return to the BBC, and some of the organisational issues were addressed by splitting the RSS committee into two, one for high-level policy, and the other for detailed intelligence. Guy Liddell continued to be frustrated that Gambier-Parry was not performing his mission regarding illicit wireless interception. In his diary on February 18, he recorded that RSS was not doing its job, as two German agents had been detected. One might interpret this discovery as a sign that RSS had indeed been doing its job, but maybe the agents – whoever they were, and whose existence was an alarming fact since T.A. Robertson had already reported that all agents had been mopped up – were not detected through electronic means. The same month he recorded that one Jean Jefferson had left the CPGB to operate a radio as an illegal, but she is not heard of again. On March 11, Liddell noted that Gambier-Parry had refused to accept responsibility for signals security. On April 1, he wrote that Frost had informed him that the Post Office had ‘bumped into’ an unknown 75-watt transmitter in Bloomsbury. It may have been SOE’s, but it all went to show (as indicated earlier in this piece) that a large amount of authorised radio transmission was carrying on of which MI5 had not been informed. And on June 3, not yet licit transmissions were detected coming from the Soviet Embassy.

The problem certainly got worse, with multiple foreign embassies now starting to transmit from the privacy of their premises, and the British government unwilling to intervene because of possible reciprocal moves. A major meeting occurred on September 10, 1943, at which (as Liddell noted) Colonel Valentine Vivian seemed ‘unaware of RSS’s charter for detecting illicit wireless communications from UK’. Liddell went on to write: “As regards the diplomatic communications of the allies there appears to be no real supervision. It was felt that to monitor and break these communications would impose too great a task on GC & CS, who were already overburdened with operational work. It was agreed that we should have a permanent representative on the Reid Committee, that we should continue to look after the security of non-service bodies, but that the results of the monitoring of the communications of non-service Govt. Depts. should be sent by RSS to the Reid Committee and not to ourselves.” Gambier-Parry’s apparent disdain for interception is shown in a record of October 13, where the head of Section VIII is shown to be a lone voice, thinking that ‘mobile units should not be taken across the Channel until RSS have detected an illicit transmitter’. The issues of quick mobility and transmission habits were obviously lost on him. (I have written more about this matter, and especially the illicit broadcasts of the Soviet spy Oliver Green, at http://www.coldspur.com/sonias-radio-part-viii/) .

Several reports written at the end of the war, in the summer of 1945 (inspectable at HW 34/18), suggest that deploying mobile units to track down illicit transmitters was a laborious and often futile exercise. (Of course, operations may have been scaled back by then, as the obvious threat had diminished, but the experiences are still informative.) In March 1945, a team of four mobile units were sent to Cheshire, and after several days managed to apprehend a GPO employee, a Volunteer Interceptor in Warrington. Another case in Birmingham was abandoned after five days. When unidentified transmissions were found to be emanating from the area of Kinross in Scotland, a troop of mobile vans was ordered from Barnet (about 400 miles away – hardly a rapid-response force) to investigate. The vans eventually discovered a Polish Military Signals Training Unit, which had conveniently and innocently continued with its traffic. Repeated interception of signals in London led back several times to the Soviet Embassy, where a ‘prototype model of a wide band DAG-1 D/F receiver’, which could track rapid changes in wavelengths used, was successfully utilised.  Such cases confirm that RSS worked under a serious lack of intelligence about potential transmitters, and it had no mechanisms for adding to the portfolio of sources of radio-waves listed above. Why was no register, with geographical co-ordinates, maintained? Moreover, the mobile force the RSS deployed was scattered so broadly as to be almost completely ineffective for trapping careful illicit operators.

The Ellipse in Kinross (from HW 34/18)

One last aspect of the interception wars is that MI5 had a respectful admiration for the Germans, believing that they were as efficient as RSS was in intercepting and interpreting traffic emanating from domestic control stations. In his diary entry for May 23, 1942, Guy Liddell describes how the Nazis were able to concentrate on Whaddon Hall (the nerve-centre for SIS, which was also handling SOE traffic, at the time), and quickly pick up the changes in frequency adopted by the British when they were communicating with agents in Europe. He concluded by writing: “It seems that the Germans have made a very close study of the form of Whaddon operators and can recognize them very easily. Their Direction-Finding apparatus is considered to be extremely good and accurate. They must think ours is very bad in view of the fact that TATE and company have got away with it for so long.” Indeed. Yet Liddell and his troops did not appear to conclude that that observation represented a considerable exposure, or that the Germans might have expected them to address this loophole as the plans for the invasion of Europe solidified.

There is no doubt more to be told of this period, but the evidence already points to a strong contrast in perceptions about illicit wireless transmission in mainland Europe and Great Britain in this period. In Nazi-occupied Europe, the organs of security moved aggressively and cruelly to eliminate any dangerous wireless traffic, although admittedly with propaganda about mechanized forces that clearly did not exist, with agents feverishly trying to escape capture by keeping transmissions short and moving around to other safe houses. In Britain, the problem was not seen to exist, but if it did, agents were able to move around unmolested in what should have been an openly hostile climate, with no safe places to withdraw to, or believed to sit at their same stations waiting conveniently for the mobile vans to turn up in a few days at the appointed time, when they would start transmitting again – and then the vans and the nervous MI5 officer might do nothing at all. Yet that is not what the RSS officers said after the war. The judgment of Hinsley and Simkins, on page 181 of Volume 4 of the History of British Intelligence in the Second world War (“In all its activities the RSS achieved a high and continuingly increasing degree of efficiency”) merits some re-inspection. The mission from Barnet to Kinross particularly epitomizes the poor use of intelligence and resources.

The Double-Cross System

After the invasion of Britain was called off by Hitler towards the end of 1940 (but kept alive for propaganda purposes until well into 1941), the role of the captured and turned wireless spies as an instrument for influencing Nazi policies was debated at length. All through 1941, and the beginning of 1942, officers of MI5 had discussed among themselves, and sometimes with outsiders, such as those in Military Intelligence proper, what the role of the information passed on to the Abwehr should be. Should it be veiled propaganda? Should it overstate or understate Britain’s military capabilities? Dick White recommended to his boss, Guy Liddell, in April 1942 that the Committee managing double agents should change ‘from that of a body of censors to that of a body of planners’, adding that ‘the difference is that we are now asking questions of the Germans while previously we were answering questions from them’.  Yet it needed a lead. It was not until July 1942, after John Bevan had replaced Oliver Stanley as head of the London Controlling Section, that operational plans were able to take on more solidity. The XX Committee, under Masterman’s chairmanship, and MI5’s B1A could start to think about serious deception strategies. (Volume 4 of the authorized History, by Hinsley and Simkins, covers this period very well. KV 4/213 at the National Archives is useful. Ben Macintyre’s breezy but uneven Double Cross is also generally recommended as a contemporary study of the project.)

Meanwhile, the Committee had to convince the Abwehr that its remaining agents were safe, and ready for action, but not over-exuberantly so. After all, the Abwehr was supposed to be in control. Long discussions took place over the necessity of passing facts on via the agents, in order to maintain credibility, but also allowing for occasional mistakes. Yet one critical aspect of the whole double-cross operation was the extent that the undeniable primary contributors to the successful deception project (BRUTUS, TREASURE, GARBO and TRICYCLE) were mostly very late arrivals to the scene. What is even more important to state, moreover, is that none of these was a classical ‘double agent’. They were all Allied sympathisers who had inveigled themselves into the Nazi apparatus under the pretence of wanting to help the Axis cause, but who then betrayed their recruiters by disclosing their true allegiance when they arrived in Britain (or spoke to British officials in Lisbon.) Admittedly, they might have been lying (and agent ZIGZAG fell into this highly complex netherworld), but MI5 strenuously tried to verify stories. TATE was the only true double agent, who had been turned after he had been captured, convinced of the necessity of his role as a tool of British intelligence, mostly out of the fear for his life, but who then gradually came to appreciate the benefits of his democratic host country. As I explained in the last chapter, TATE’s value as a contributor to the deception over FORTITUDE was diminished because the necessity for him to find a modus vivendi and occupation to survive in Britain forced him to be a more reclusive and less mobile observer of invasion preparations.

For a short while in April, 1942, moreover, the Double-Cross Committee had considered the implications of running double-agents overseas, and taking over the transmitters that SIS maintained at Whaddon Hall. This was because the SOE agent VICTOIRE, Mathilde Carré, who claimed she had escaped from her German captors, had convinced her interrogators that she was genuine. Masterman and Marriott in B14 thus started to plan how messages could be sent back to members of the Interalliée as a method for deception, since MI5 and SIS knew that the agents had been turned by the Germans, but the Germans were assumed not to know this. The task presented fresh challenges as to how lies and truth should be managed without detriment to the real war effort. Before this task became reality, however, VICTOIRE was unmasked by one of the officers she had betrayed, agent BRUTUS (see below), and she was incarcerated for the remainder of the war.

In any case the official accounts need to be treated carefully. John Masterman’s Double Cross System contains an Appendix that claims that there were at least 120 double agents managed by the XX System, and it lists thirty-nine of ‘the more interesting cases that were operated from this country’. Yet this list includes such dubious characters as SNOW (who was dropped as early as March 1941 since he was probably a triple agent), the enigmatic GANDER (who may never have been turned, and disappeared mysteriously from the scene in November 1940), and the turncoat SUMMER (who tried to escape in January 1941, and whose fate remains controversial). It also includes such figures as BALLOON, who was recruited by TRICYCLE, which hardly puts him in the class of ‘double agent’: the term sometimes used in the authorised history by Hinsley et al., ‘double-cross agent’, is more suitable. (Masterman omits to mention a figure named BRISTLE, the cryptonym appearing in KV 4/214 at the National Archives, an oversight that suggests there may be a yet undiscovered tier of ‘less interesting’ agents whose names MI5 would prefer to forget.) As Hinsley and Simkins more accurately represent the state of the game in late 1943: “The newly acquired double agents [sic!] off-set the loss of Zigzag, Rainbow, Father, Dragonfly, Balloon and Mutt and Jeff, whose operations were now closed down or suspended.”

This account necessarily focuses on agents who successfully contributed to deception through wireless communications, which was a complex issue in its own right. Because of MI5’s desire to have information passed quickly to the Abwehr, agents who had hitherto used secret ink or microphotography requested wireless apparatus from their controllers. Indeed, GARBO exploited a delayed, but highly accurate, message about TORCH landings, which conveniently arrived after the event, to encourage a move to wireless usage. This may have prompted the Germans to accelerate the use of wireless communications with GARBO. That would, of course, allow the British to get disinformation in the hands of their adversaries in a much more timely fashion, but it would also eliminate the convenience of delivering highly accurate information with a built-in delay, thus increasing the risk of injurious retaliatory action. The adoption of radio did necessitate the delivery of codes, however, which was mightily useful for GC&CS in extending the range of intercepted signals that could be decrypted.

So how did these vital agents fare in the use of radio? The final 1942 entry in the files of TATE [Wulf Schmidt] at Kew expresses confidence that the enemy trusts him, and that his story about transmitting early in the morning, before the farm hands go to work, has been accepted. Yet 1943 appeared not to be so successful, and his handlers voiced concern about his viability.  (It was impossible to verify what the Abwehr thought of him, as messages from Hamburg to Berlin were sent by land-line.) During the period March-September he received only fourteen messages from the enemy, most of them very routine, as if it could not expect much valuable information from an agent fully engaged in agricultural work. An added complication arose because of the repatriation of a Nazi in November 1943. It was feared that this officer might have picked up rumours inside the camp where he was being held to the effect that MUTT, JEFF, SUMMER and TATE were all under control of the British.  That encouraged MI5 to put TATE on ice for a while. A report in early January 1944 also lamented the fact that he had only one transmitting frequency (4603 kcs), which made communication as far as Hamburg difficult outside daylight hours. TATE thus made a request to have a small portable apparatus workable off the mains, and the minor role he was able to play in OVERLORD will be described in the next episode.

Agent BRUTUS

BRUTUS [Roman Czerniawski], a former Polish fighter Pilot, experienced a comparatively short career as a double-cross agent. After the Germans arrested him in late 1941 in France, where he had built up an intelligence network, he manufactured a deal whereby he traded the safety of his family for a role spying in Britain. Before he left Paris, he was given quartzes to take with him for the purpose of building a transmitter with the help of his Polish friends, although BRUTUS asserted that it would be difficult finding a wireless operator. After an ‘escape’ via the Pyrenees, he arrived in England on October 2, 1942. Certain necessary checks with Poles in exile complicated his adoption, but he was approved, and established contact in December 1942, with an apparatus constructed for him by MI5. (The archive does not indicate how he suddenly acquired operating skills.) He was then instructed to build his own radio set in early January 1943. Masterman was cautious, telling Bevan he wanted to run BRUTUS giving information, not as a deception medium.

The year 1943 turned out to be problematic, as BRUTUS stumbled into hot water with the other Poles over the Katyn massacre, and his overexuberant politicking. (The Germans had discovered the site of the massacres in April, but the Soviets had denied any responsibility, thus causing a rift in Allied circles. On April 25, the Soviet Union broke off relations with the Polish government-in-exile.) Moreover, there was a security problem, as the Poles had access to BRUTUS’s codes (and thus might learn about the deception plan for OVERLORD). Harmer also reported to Robertson on May 6 that White and Liddell were concerned lest the Russians intercept and decode the BRUTUS traffic and use it ‘as a basis for their allegations that the Polish Government are maintaining contact with the Germans’. Reed assured Harmer that the range for BRUTUS’s transmitter was only 400 miles, so there was no danger of interception, but the episode showed the tangled politics that were starting to affect counter-espionage exercises. BRUTUS successfully reported on the arrest of CARELESS in May 1943, and Ultra decrypts showed that his reports were being taken seriously. However, BRUTUS’s arrest in the fracas over Katyn caused an awkward interruption. MI5 found him a notional ‘operator’ (purportedly in Reading, actually working in Richmond, thus apparently breaking the rules observed in other cases to protect against German direction-finding) so that he would not have to operate the wireless himself. Intercepts indicated that he was not fully trusted, and by the end of the year, Harmer was suggesting that he be used solely as a courier. On the last day of the year, however, BRUTUS informed his handlers that he needed a new transmitter.

Agent TREASURE

The career of TREASURE [Lily Sergueiev], a journalist of Russian extraction, was very short, and she was not even activated as a wireless agent until January 1944. Yet her association with German Intelligence went back as far as 1937, when she had declined to work for a contact in Berlin, one Felix Dassel. After the fall of France, when in Paris, she had recontacted Dassel, and agreed to work for the Abwehr. She had been introduced to her handler, Emile Kliemann, in June 1941, and soon started receiving training on operating wireless equipment. This was somewhat unusual, as the Abwehr seemed keener at this time to have their agents use couriers, secret writing and microdots. By February 1942, she had started practicing, transmitting and receiving on a proper set, but for reasons primarily to do with Kliemann’s rather erratic behavior and complicated love life, the practice was neglected. Indeed, as late as May 18, she was taught how to use invisible ink, and it was not until July 17, 1943 that she appeared at the British consular office in Madrid declaring that she intended to travel to England to spy, but wanted to switch her allegiance.

After researching her background, MI5 concluded that her intentions were genuine. But she still had to wait for the distracted Kliemann to get organised, and it was not until a few months later (her MI5 handler, Mary Scherer, said November 11; Ben Macintyre states October 7) that she was able to fly from Gibraltar to Bristol. Kliemann had promised her that she would be given a wireless set to be disguised as a phonograph, but he let her down, unable to procure one for her, instead promising that she would be passed one after she arrived in Britain. She boarded the plane without it – also without her beloved dog, an incident that would later cause deep rifts between her and those in MI5 she trusted. Her activity as a spy was then further delayed owing to her becoming seriously ill in December, and being hospitalised. Thus it was not until January 11, 1944 that MI5 started conceiving plans for putting TREASURE in possession of a wireless set. She was able to write to Kliemann informing him that she had now bought an American Halicrafter radio (actually supplied by MI5), even though possession of an unlicensed wireless transmitter was still a civil offence.

Agent TRICYCLE

For most of his career TRICYCLE [Dusko Popov] was not a wireless agent, and he never used such equipment himself. He had managed to convince the Germans as of his bona fides, while remaining free to travel because of his import/export business, but had declared himself to the British back in 1940. Yet he had been sent by the German to the USA in October 1942, and spent most of 1943 in what turned out to be a fruitless (and expensive) sojourn. Even before his spell in the USA, the British had deciphered messages that indicated that the Germans had suspicions about him, but TRICYCLE bravely walked back into the lions’ den in Lisbon, and managed to brazen out his interrogators, who were anxious to believe that they still had a valuable resource under their control. On September 14, 1943, TRICYCLE flew back to Britain, carrying with him various espionage material and money, and also a wireless transmitter. So who was to operate it?

TRICYCLE had ingeniously convinced the Abwehr of a scheme to infiltrate supposed Yugoslavian Nazi sympathisers into Britain, disguised as refugees. Through his brother, Ivo Popov, TRICYCLE arranged for a naval officer called Frano de Bona to be recruited by the Abwehr and trained as a wireless operator. TRICYCLE returned to Lisbon and Madrid in his role as a Yugoslav diplomatic courier in November 1943, and there negotiated de Bona’s [FREAK’s] passage via Gibraltar to London, where he would operate TRICYCLE’s equipment. On December 8, Guy Liddell recorded his fear that the whole TRICYCLE set-up might collapse at any moment, but later that month FREAK started his work as a wireless operator. He would transmit regularly (his location not apparently revealed) for five months until being necessarily closed down because of a scare.

Agent GARBO

The most famous of the double-cross agents, and the one who contributed most to the deception exercise of FORTITUDE, was the Spaniard Juan García Pujol (GARBO). Again, his career went back a long way, and it was not until late in the war that his ‘network’ was supported by wireless transmission. He had originally presented himself to the British Embassy in Madrid in January 1941, but was turned away. Inventing information for the Abwehr, his reports were picked up by SIS, and he was eventually interviewed again in November 1941. He was smuggled out of Lisbon to Gibraltar, and hence to London, where he arrived on April 24, 1942. After interrogation, GARBO was transferred to the control of B1A in MI5. Over the next few years he would craft hundreds of letters written in secret ink, which mysteriously managed to reach the Germans in Spain and Portugal. As Ben Macintyre writes: “The information they theoretically supplied was written up in secret ink and dispatched inside innocuous letters that the Germans believed were either brought by courier or sent by airmail to various cover addresses in neutral Spain and Portugal. In fact they were transported in MI6’s diplomatic bags.”

Yet this was not going to be a swift enough medium for the purposes of FORTITUDE. In August 1942, GARBO had in principle gained permission to use wireless. The Abwehr had encouraged GARBO to make his ‘notional’ agents use secret ink to communicate directly, which would have made the control and distribution of disinformation very difficult. Thus GARBO, having fortuitously ‘discovered’ a radio technician employed on the outskirts of London who was a friend of his ‘Agent No 4’, suggested that wireless should now be attempted for communications. When GARBO reported, in November 1942, on convoy departures for the TORCH landings, and the information arrived too late for the Germans to act upon it, it was a timely signal for them to adopt a newer technology, and they wrote to him on November 26 more warmly accepting his recommendation. In the words of Hinsley and Simkins: “To begin with a large volume of material continued to pass by air mail and courier. From the end of August [1943], however, almost all his [GARBO’s] messages were sent on his radio link. This followed from the need, in support of Allied deception plans, to force the Germans’ correspondence with him on to the air and receive it with greater speed, and also from the fact that, to give verisimilitude to his network by indicating to the Germans that MI5 was aware of its existence but could not track it down, steps were being taken to show them that its air mail letters were being intercepted.”

The first transmission was scheduled to take place on March 6, 1943, and Guy Liddell reported that GARBO did in fact establish radio contact with Madrid on March 12, with the MI5 operator resident at 55 Elliot Road, Hendon. The provision of a new cipher by the Abwehr was highly valuable: Liddell further commented, on June 5, that GC&CS regarded the results of interception as ‘outstanding’. Yet wireless procedures were outstandingly undisciplined. Despite instructions to their new operator to keep messages as short as possible (‘No transmission should exceed fifty groups for safety sake’), and warnings about direction-finders, even referring to the use of aeroplanes (which was a technique the Abwehr was domestically familiar with), GARBO’s operator was shown to be on the air for two hours at a time in June 1943, owing to the prolix and flowery reports that he and Tomás Harris, his minder, compiled. By the end of August, nearly all GARBO’s messages were sent by the wireless link, and after one or two hiccups due to the Abwehr’s concerns about British censorship of the mails, and possible exposure of the wireless-led network, communications flourished for the remainder of the year.

As an interesting sidenote on the efficiency of RSS, Hinsley and Simkins report that the service was able to detect GARBO’s station. It was clearly closely involved with tracking the transmissions of the agents. What had happened was that GARBO had been given a transmitting plan that required the station to adopt military procedures for callsigns and introductions, with the result that the signals would be confused with a swelter of other military traffic, making connection with Madrid difficult for a while. “  . . . in fact GARBO’s transmissions were temporarily lost by the operators who had been intercepting them for the RSS from places as far apart as Scotland, Gibraltar and Canada. . . “, the historians wrote. “It was a tribute to the efficiency of the RSS’s intercept network that after a few weeks it again reported Garbo’s transmitter as a suspect station.”

Conclusion

As the preparatory period for the long-awaited invasion of Europe started, a strange, asymmetrical confrontation of wireless intelligence had developed. From the German side, the notion of a powerful direction- and location-finding apparatus had been created in response to a pervasive and potentially dangerous threat. Yet it was hard to implement. Its menace was used more as a deterrent than an enforcement mechanism, the security organs struggling with the practical limitations of such techniques, and having to rely more on informers and infiltration to subvert and destroy the enemy’s networks. In Britain, a similar powerful detection capability kept a close ear on the airwaves. The authorities, however, confident that no genuine hostile agents were operating on native soil, owing to the RSS’s interception, and GC&CS’s decryption, of Abwehr traffic, maintained a surprisingly casual stance towards illicit transmissions and their origin. Both German and British Intelligence were justified in thinking that the capabilities of their foe were at least as advanced as their own. After the war, the British boasted of their capabilities in a manner similar to that of the Germans. Yet MI5, in managing its Double-Cross System, was woefully careless in supervising the transmission schedules of its agents, and the Abwehr deluded itself in thinking that its agents could survive undetected in a small, hostile island.

New Commonplace entries can be found here.

Leave a Comment

Filed under Espionage/Intelligence, General History, Politics, Technology

The Mystery of the Undetected Radios (Part 4)

“A masterpiece of Radio Precision”? see below

News Update:

Alert readers will have noticed that I received important communications from Roland Philipps (the biographer of Donald Maclean) and from Jan-Willem van den Braak (the biographer of the Abwehr spy Jan Willem ter Braak), whose work is being translated from the Dutch for publication in the UK. I shall report on the outcomes of these dialogues in next month’s report.

An observation on Guy Liddell and Roger Hollis by one of my contacts in intelligence inspired me to break out in verse on the subject of MI5’s efforts to counter Soviet influences. The doggerel can be found at DiaryofaCounterEspionageOfficer.

************************************************************************

After I had put Part 3 of this saga to bed at the end of September, some thoughts that I had vaguely touched on in earlier episodes returned to me with more vigour: What if the mistakes over ter Braak and the controversial report by Walter Gill (which effectively concluded that domestic wireless interception was not necessary) were both deliberate exercises by MI5 and its partners? Were the plans for the double-cross operation that far advanced in the last few months of 1940 that it was considered vital to give indications – in the belief that the Abwehr would pick them up – that Britain’s wireless interception policies were so weak that German agents could essentially roam at will, and broadcast home undetected? After all, as early as September 1939, Guy Liddell of MI5 had written that ‘it was in our interests that the Germans should regard us as grossly inefficient in these matters’, and that ‘if they thought our organisation was good they might well ask how it was we managed to get his [SNOW’s] messages through’. And were the Abwehr’s planting of obviously fake identification cards on its agents a deliberate ruse to determine how gullible the British counter-espionage services were?

These may be utterly fanciful notions, but they have a modicum of sense about them, as all such exploits at face value are very difficult to explain. One has to assume that agencies like MI5 and the Abwehr were continually thinking: how will our enemy counterpart think and act? (A British FOES committee did in fact exist: Guy Liddell described it as ‘an inter-services committee that tries to put itself in the position of the enemy intelligence service’.)  And, if some sensible insight were applied, each intelligence section should have assumed that its counterpart, because of native influences, might in some circumstances act in a different fashion. Thus, in this instalment, I start to explore the variations in the strategies and successes of the major European-based espionage/sabotage organisations: SOE (Special Operations Executive), the German Abwehr, and the network of the Soviet Union’s GRU and KGB spies, and what their controllers should have learned from their experiences in one theatre of war to apply to another. There is a symmetry in some of the things undertaken by each organisation, as they strain to develop measures to confound the forces trying to counter them. Yet one can also spot asymmetrical aspects, driven by the idiosyncratic nature of each force, including their overall motivations and objectives, the personnel they selected, the territorial dimensions, and the cultural drivers behind their operations. It is hard not to suppose, however, that the policies of each were not somehow affected by their knowledge of what their adversaries were doing with their own offensive activities.

The focus of my research in this series has been the detection of illicit wireless. It is worth recording here that the primary purpose of what is commonly known as RDF (Radio Direction-Finding, but implicitly including Location-Finding) had, before the war, been the interception and decryption of government (e.g. military, diplomatic and police) traffic. Initially, precise location was not as important as content. As countries started to perform intelligent traffic analysis, however, the origin – and mobility – of transmitting stations, especially military units, became much more significant, often providing intelligence even though the underlying messages could not be decrypted. Then, as the combat started, organisations had to start to apply their knowledge to the possible threat of illicit stations operating behind their own lines.

With all three combatants, the techniques for long-range triangulation were well-developed by the time war broke out, and thus could in principle be quickly adapted for identifying illicit domestic transmissions. The paradox was that, owing to the vagaries of the behavior of radio waves, it was often easier to pick up transmissions originating abroad than those issuing from inside the country’s boundaries. As I explained in Part 1 of this saga, low-powered wireless sets operating on high-frequencies in domestic territory, designed to exploit ‘bouncing’ off the ionosphere, were often hard to detect because of the skip zones involved, and widely dispersed human interceptors would have been needed to pick up their ground waves. Such a set-up was possible in the United Kingdom, but not in the expanding German Reich. Moreover, the finer granularity required for locating individual wireless sets (at building-block or house level) demanded new mobile equipment and techniques not explored in long-range location-finding.

As I discuss the strategies and challenges of the three espionage forces, and attempt to assess their effectiveness, I shall be considering them under the following criteria:

  1. Operational leadership: How good were the directors in planning how objectives should be met, and following up by providing the motivation, material, and structure to allow agents to be successful?
  2. Quality of operators: Were agents with the appropriate profile chosen for the job in hand?
  3. Quality of training: Did the agents receive thorough and suitable training?
  4. Quality of equipment: How effective was the equipment (primarily wireless apparatus) for the location of operation and for transmission needs? Were conditions such as local power supply properly taken into account?
  5. Operating procedures: Were safe and secure operating procedures defined, and did the agents follow them?
  6. Remote support: Did the agents receive reliable and effective support from their home controllers?
  7. Detection capabilities: How effective were the enemy’s radio-detection and direction-finding mechanisms?
  8. Social environment: How hostile or sympathetic was the social environment in which they had to work?
  9. Counter-Intelligence strategy: What goals drove the counter-espionage strategy of the enemy on whose territory the spying took place?

June 1941 constitutes the major chronological dividing-line in the conduct of wireless espionage. (In the light of my research, I have deviated from the temporal Phases identified in my first post in this series, which had Phase 1 completing at the end of 1940, and Phase 2 winding down in June 1942.) The Nazi invasion of the Soviet Union immediately changed the German attitude in Soviet counter-espionage from one of wary passivity to aggressive pursuit. The Russian stance in illicit communications switched from cautious dormancy to careless urgency. For Britain, it signalled that any planned invasion of the island nation had been postponed indefinitely: the timing coincided with the transfer of RSS to SIS, and the implementation of the new structure in MI5 under David Petrie. The date has less significance for SOE: it was still in an experimental, groping stage in the summer of 1941, with only two radio-stations established in France by that time. My analysis thus presses forward in this dimension of espionage and sabotage to address the continued struggles of the unit into 1942. I now summarise the activities of the three agencies in this period before delving into more detail.

I have shown how the greatest intensity of Nazi attempts to infiltrate British territory occurred in the autumn of 1940 (Operation LENA), with a couple of reconnaissance landings (by Jakobs and Richter) occurring in the spring of 1941 – i.e. before Germany’s alliance with the Soviet Union turned into a clash. By then, with the plan to invade the United Kingdom abandoned, and Hitler’s attention now directed to Operation Barbarossa, the agents whom the Abwehr had apparently successfully installed in Britain took on less importance. They appear to have been largely forgotten, or abandoned, and it took the arrival of new ‘spies’, such as TRICYCLE, GARBO and TREASURE (whom I shall cover in the next chapter), to re-activate the espionage – and the Double-Cross – project. Yet using wireless was not at the forefront of the Abwehr’s plans, and MI5, in their efforts to facilitate the passing on of fake information, had to be very careful and imaginative when encouraging use of the medium.

As far as Britain’s own plans for espionage and sabotage were concerned, Churchill had in the meantime (July 1940) established the SOE as a force to penetrate Nazi-occupied Europe, and to soften up and harass the invader’s government of occupied territories. Yet this was not primarily an espionage organisation, like SIS (whose network had been almost completely destroyed at the outset of war.) It was an outfit committed to sabotage, and, while wireless communication became a critical part of its operational infrastructure, the technology was used more to arrange for shipments, drop-offs, and pick-ups, and only secondarily as a mechanism for providing intelligence. Sabotage operations also drew more obvious attention from the enemy: furthermore, in the first two years of its existence (i.e. until the summer of 1942), SOE was hampered by being reliant on Section VIII of SIS for its wireless equipment, wavelengths, codes, etc. The experience in responding to illicit SOE transmissions in France may have given the German counter-espionage agencies a leg-up when the Soviet apparatus fired up in the summer of 1941, but, as will be shown, the evidence for this is shaky.

When Hitler invaded the Soviet Union, all Soviet agents in place in Germany were immediately activated to provide intelligence about Nazi war-plans. Yet they had not been completely dormant before then. The situation was in fact more complex than that. After the show-trials and purges of 1937-1938, the KGB and GRU networks had been patiently rebuilt – not just in Germany, but across most of Western Europe. As early as May 1940, however, when Paris fell, Moscow suspected that relations with Nazi Germany – despite the Ribbentrop-Molotov Pact – might deteriorate, and diplomatic representatives (e.g. Kobulov in Berlin) started building networks of informers, not only in Germany but also in Poland and Czechoslovakia. Elsewhere, the Soviet Union’s spies had long been active, such as in the origins of the famous Red Orchestra group in Switzerland, led by SONIA (Ursula Kuczynski) and DORA, the Hungarian Sándor Radó, who had been recruited in 1935, and moved to Switzerland in 1939. Before 1941, however, couriers, and communications through local Soviet embassies, had been a much more convenient method of passing information than the use of wireless transmission methods.

Abwehr Spies up to June 1941

Admiral Canaris of the Abwehr

The decision to infiltrate spies into Great Britain in late 1940 was taken at short notice, but, like many events of a time when feints and deceptions were part of the strategy, the exact date when Admiral Canaris initiated the LENA programme is uncertain. In 2018, Bernard O’Connor, relying on the rather dubious transcription of Lahousen’s War Diaries claimed by Wighton & Peis sixty years earlier, asserted that Canaris told his Abwehr officers as early as June 22 that gathering intelligence on Britain, in preparation for the planned invasion, was of the highest priority. That early preparation is vaguely echoed by Niklaus Ritter in his 1972 memoir, Deckname Dr. Rantzau, where he improbably describes being in the company of Caroli (SUMMER) and Schmidt (TATE), ready for their departure some time in July, when they had already completed their eight-weeks’ training. Yet Ritter’s memory was at fault: he describes them as leaving on the same plane – something which the British archives strongly refute, so one must question the reliability of his memory. John Lukacs, in The Duel, represents Admiral Raeder as still trying to talk Hitler out of invading Britain as late as July 11, with Hitler responding in terms of wanting to make peace with the United Kingdom. O’Connor and Ben Macintyre both refer to a conference held in Kiel ‘some time in July’ to plan the details of the LENA operation, an event confirmed by the Kew file on the Hamburg Abwehr officer Praetorius (KV 2/170-1), and given precision by KV 3/76, which sets it as taking place on July 16. That would dovetail with Ritter’s account that eight weeks of training had to be accomplished to meet Hitler’s deadline of September 15. 

Praetorius’s recollection was that the agents parachuted in at this time would ‘only have to be of independent means for 6-8 weeks as by at time the invasion of England was expected to be an accomplished fact.’ Yet the chronology does not work. If a decision had been made in July, the recruitment and training of agents was supposed to take eight weeks, and their subsequent independent existence on British soil might have been expected to take another six to eight weeks, the latest date for a successful invasion would have to be placed as late as early November. While Anthony Cave-Brown gave August 1 as the date that Hitler issued his Directive 17 to prepare for the invasion of Britain, Operation SEELÖWE (SEALION), Churchill himself reported it as being on July 16, with Hitler’s apparent objective of having his forces arrive four weeks later. On September 11, however, Hitler had to delay the invasion order until September 24, and on September 17 he ordered the indefinite adjournment of SEALION, and formerly cancelled it on October 12. Yet the first LENA agent, Caroli (SUMMER) did not parachute in until September 3, and his colleagues were still arriving in early November. It sounds as if Canaris gave Hitler unreasonably optimistic indications of the speed with which agents could be recruited and trained: if Hitler had been able to stick to his original plan, there would have been no planting of infiltrators in the United Kingdom, successful or not, to assist the invasion. Yet the program unaccountably went on after invasion plans were suspended, which would have made nonsense of the ability of the agents to survive independently for a few weeks.

Given the haste by which recruits had to be selected, vetted, and prepared, it is thus difficult to take seriously the claim made a few years ago (in Monika Siedentopf’s Unternehmen Seelöwe) that the invasion of Britain was sabotaged by Canaris and his team, in that they selected unsuitable candidates as spies who simply let the side down. Apart from the chronological problems listed above, however successful the few who landed might have been in evading capture, their effect on a planned invasion that required destroying the Royal Air Force would have been minimal either way. But that does not mean that the Abwehr’s project was not quixotic, or even cruel. The agents were chosen in a hurry: they were not native Germans, but mostly citizens of bordering countries (Denmark, Sweden, the Sudetenland – the last, of course, transferred from Czechoslovakia to the German Empire). Some were diehard Nazis, some were lukewarm, others were pressured into signing up by threats. The belief was that agents from outlying countries would fade into the background more easily than native Germans: some had spent time in the UK beforehand, but, overall, they were hopelessly unprepared for life in the United Kingdom. And as potential observers, they were untrained. Reports at Kew indicate that ‘though they were expected to report on such military objectives as aerodromes, land mines and gun batteries, on examination they showed only a vague idea of the significant points to note.’  They had ‘only an amateur knowledge of transmission technique.’

The main point, however, was that the spies of the LENA operation were not expected to be operational for long, a fact that is reinforced by the way that most of them were equipped. More than half of the eighteen (the exact number is debatable) who landed, either by parachute or boat, between September 3 and November 3, 1940 either carried with them a transmitter only, or no wireless equipment at all. A transmitter might have been useful for sending a brief set of dazzling reports about air defences, bomb damage, or weather conditions, but without an ability to have confirmed whether one’s messages were being received correctly, it would have been a short and demoralizing career. For those agents being parachuted in, wireless apparatus was a significant health hazard: at least two spies were injured by virtue of their collision with the earth when harnessed to sets weighing twenty pounds or more. Most had not practiced a parachute-jump before. Moreover, many were told in Hamburg that there was not enough shock-proof material available, and thus they would be equipped with transmitters only. If wireless sets were dropped separately, there was the risk of the apparatus’s never being found. TATE demanded he be equipped with a combined Transmitter/Receiver. As his Kew file reports: “His controller, RITTER [Captain Rantzau] then informed him that arrangements were being made for him to take with him to England a separate transmitter and receiver and also a large transmitter (called a ‘Z.B.V.’) which would be dropped separately and which he could destroy if the smaller sets were unbroken after landing.”

MI5’s analysis of the equipment the agents were provided with would indicate that they did not have a high chance of success in trying to contact their controllers. The boat agents (Meier, Waldberg, Kieboom and Pons, who arrived on the Kent coast) were equipped with compact and light cases, one weighing 7 lb., and containing batteries and connecting wires, the other weighing only 4 lb., containing the transmitter, aerial and spare valve. (This was in dramatic contrast to the bulky devices that SOE agents were required to take to France or, say, Yugoslavia, in following years.) Yet the experts judged that such low-powered devices ‘would require exceptional conditions to work over 100 miles’, with an expected range of nearer 50 miles. *  If that assessment is correct, it would show an extraordinary misjudgment by the Abwehr experts: reducing power to such a degree that transmissions would not only be undetectable locally, but would also not have enough energy to reach their intended target. This statistic is put into perspective by the fact that the distance between the port of Southampton and Cherbourg is over 100 miles, while German wireless agents were transmitting home from as far afield as New York and Brazil.

[* This opinion needs to be balanced against that of E. H. Cookridge, who, in his 1947 work Secrets of the British Secret Service, described Kieboom’s equipment as ‘a masterpiece of radio precision’, following up by claiming that ‘the transmitter allowed to send [sic] messages over a range of more than 600 miles, yet was so small that it could be hidden in two leather boxes  . . .’ (see Figure below). In his Preface, Cookridge thanked the Foreign Office, the War Office, the Home Office and the Lord Justice’s Office for their assistance, so his book should probably be regarded as an item of selective disclosure for propaganda purposes, perhaps maximizing the wireless threat.]

SNOW’s transmitter was reported to have a much more realistic range, of up to 1200 miles.  Likewise, CAROLI’s (SUMMER’s) equipment was much heavier and more powerful, but would have a corresponding disadvantage of requiring much more space to set up the aerial. “Aerials provided would not be easily untangled and satisfactorily erected except in secure privacy with plenty of space. E.g. indoor space 60 ft. long or a secluded wood with a fairly clear space 6o ft. long with trees etc. on which to tie the end of the aerial to a height of at least 6 ft.” How a spy in tight wartime conditions, in densely populated England, was supposed to accomplish such a task is not clear. A tentative conclusion by the report at KV 3/76 was that the agents were so ill-prepared that they should perhaps be considered as decoys.

Kieboom’s equipment details (from Cookridge)

Nevertheless, it seems that the Abwehr stations stayed observant, looking for transmissions from the agents. The same file, K 3/76, based on interrogations of the six prominent spies captured by September 1940, supplemented no doubt by RSS interception and decryption of Abwehr exchanges, discloses the following: “It appears from other sources [sic: surely a code for Ultra decrypts] that a constant watch is kept by Hamburg, Berlin, Paris and Cherbourg, for the reception of any wireless messages by all agents despatched to the U.K.  This is presumably in order to make sure that messages shall not be missed through bad atmospheric conditions.” The advantage gained by the German Reich’s territorial extension into Northern France (which also aided triangulation for location-detection) was counterbalanced by the fact that ENIGMA radio communications had to be used rather than highly secure land-lines, which allowed British Intelligence to tap into the plans and processes of the Abwehr. Moreover, by this time, Hamburg (which would have had secure contact with Berlin) was shifting its attention to Norway, placing the responsibility for Britain on to Paris and Cherbourg. A dangerous increase in interceptible traffic was caused by the fact that the Abwehrstelle in Brussels was used as an intermediary point for traffic, with messages passed to it from advance stations to be decrypted, and then passed on to Hamburg, Paris, or Berlin.

Because nearly all of the spies were picked up soon after they landed, little can be said about the adequacy of their training. Ter Braak apparently struggled with his receiver: concealing aerials in densely-populated Britain, with vigilant landlords and ladies, would have been a problem. TATE had only one frequency to work on, which was effective only in daylight hours: this inhibited his activity later. TATE admitted that he had been taught the fundamentals of operating, but nothing about wireless theory, which would mean he would be helpless when problems occurred. He said that he only knew “the practical details of how to join it up, erect the aerial, and tune the transmitter by the lamp. He thought he could spot a disconnected wire inside, but that was about all”. As Reed of B1A reported: “He had been instructed to join motor-cycle batteries in series, but three 6 volt batteries would burn out his valves.” Consequently, even with MI5 assistance, TATE struggled to make consistent contact. Reed reported, on October 1, that ‘experiments with [TATE’s] wireless were unsuccessful due to inefficiency of aerial provided with a set of so small an output.’ His first successful message was not sent until October 10: he was supposed to send a postcard in invisible ink to a contact in Lisbon if his wireless failed to work. She never received the postcard.

TATE had quickly understood that his life depended upon abandoning his Nazi affiliations, and following the instructions of his new captors. Unlike SUMMER, he did not have second thoughts, and thus did not employ any security code to indicate that he had been turned. (He claimed that the possibility of being captured and used had never been acknowledged by his trainers, and he thus did not have such a code.) He initially operated his set himself, and thus displayed a consistent ‘fist’. Yet the overall message to be gained from this exercise is that the Abwehr controllers soon lost interest. As early as September 7, Field-Marshal Jodl told the Abwehr to open up operations against the Soviet Union. The realization that German could not dominate the skies above Britain, and that a winter invasion across the Channel would simply be a recipe for failure, had by then convinced Hitler that it was time to turn his attention to the East.

What TATE’s files at the National Archives show is the enormous lengths to which MI5 and RSS went to experiment with his apparatus, attempting to make contact with Wohldorf. While SUMMER’s set had been shown to work quite quickly, MI5 provided their counterparts at RSS with all the details of call-signs, frequencies, and times so that the location-finding network of interception towers at Thurso, St Erth, Gilnakirk, Sandridge, Cupar and Bridgewater could gauge the strength of the signal, and give back advice. Hughes (W6B) and then Reed (who was on secondment from the BBC) had to move the set around from city to countryside, change the length of the aerial and fine-tune its alignment, and also have the complex instructions for TATE’s back-up set translated before they were able to send transmissions of consistent quality. Yet they were already sensitized to the need to avoid German direction-finding – to a degree that was unnecessarily cautious: they believed that the transmissions could have been localized to an actual building (e.g. Latchmere House), a degree of accuracy way beyond what the Funkabwehr was capable of at that time.

Meanwhile, agent SNOW (Arthur Owens) was being kept in close confinement. It should not be forgotten that SNOW was the original Abwehr agent equipped with wireless, and was notionally active right up until April 1941. Yet the first experiments with wireless were haphazard: he was supplied with a clumsy and reliable transmitter (only) in February 1939, but, since he was able to meet his handler, Ritter, in Hamburg until war broke out, and, after that, arrange regular rendezvous in the Netherlands and in Belgium until the Nazis overran those countries in May 1940, the use of wireless to pass on intelligence was not so critical. Of course, that made the task of monitoring what he said impossible, and suggestions that SNOW had betrayed his country by revealing suitable targets for bombing (i.e. going beyond the ‘chickenfeed’ that he passed in his encrypted messages) caused MI5 to terminate him, and incarcerate him for the remainder of the war.

Agent SNOW

MI5 was aware of SNOW’s wireless usage from the day his set was picked up. SIS even broke the set, and had to repair it. But SNOW did not make his first successful transmission until late August 1939: soon afterwards, MI5, aided by his wife’s jealous reporting of his duplicitous activity, arrested him, and then found both his transmitter, and then a receiver, concealed at his property in Surbiton. Under MI5’s tutelage, SNOW moved house to premises where his aerial would not stand out so obviously, and transmitted regularly on weather and less than critical military operations and preparation. The first Double-Cross message was sent on September 9, but no confirmation of receipt occurred for some weeks. At some stage in October, Maurice Burton, who had earlier checked to verify that SNOW was transmitting as instructed, took over the operation of the apparatus, and eventually a new afu transmitter-receiver was delivered through a third party.

Whether the Abwehr had been careful enough to pay attention to SNOW’s radio ‘fist’, or whether Burton was adept enough to emulate it, is not clear. The archival reports give every indication that Robertson and his team assumed that Ritter must have concluded that SNOW was being controlled by MI5. Guy Liddell even wrote, on February 2, 1941: “Another point that occurs to me us that the Germans must now be wise to the game of collaring an agent and forcing him to use his wireless set in our interests. There is in fact evidence that they are doing it themselves.” Yet the Abwehr used what SNOW fed to them concerning passports and ration cards to supply the LENA agents, and lure them to their doom or glory. Exactly who was deluding whom by the time SNOW was regarded as a high security risk may well never be established. A triple agent works only for himself, trying desperately to play one employer against the other in order to survive. Interrogators of Ritter after the war concluded that he had realized that SNOW had been turned, but, when Ritter wrote his memoir in 1972, he gave no suggestion that SNOW was anything but the genuine article. Ritter believed that SNOW was being used by MI5, but that the Abwehr had outwitted them. He certainly would not wanted to have admitted to his bosses in Berlin at the time that he had been deluded. Other Abwehr officers interrogated were more outspoken and direct about their suspicions: I shall explore these in a later chapter.

MI5 and RSS gained much from these experiences. They learned about the enemy’s equipment, and the RSS was able to test out its interception and location-finding techniques when they applied their sensors to TATE’s transmissions, in order to evaluate how effective they were. Yet this was a precarious time for MI5: the seeds of the successful XX Operation were quickly sown, but Liddell and others also came to realise that allowing ‘undetected’ radios to operate would require the existence of a ham-handed and inefficient detection service for them to evade interception. This concern would continue to dog MI5 throughout the war –  the fear that the Germans must assume that the wily British had better radio-detection finding equipment than appeared to be the case, and would thus assume that their agents were not operating freely. And, as I pointed out in my article on ter Braak, is it not somewhat ridiculous to think that, in densely-populated Britain, with a citizenship well advised to look out for suspicious activity, that an obvious foreigner, with accented English, could traipse round the country picking up information, and then return to some lodging where he managed to conceal the existence of a lengthy aerial while sending in his reports?

For the Abwehr, their LENA spies were dispensable. The espionage service did not think they would survive long, and it had low expectations of their deliverables. As a July 1944 report submitted jointly by MI5 and SIS declared: “According to the calculations of one Abwehr officer, eight-five per cent of the agents dispatched were never heard of again; ten per cent turned in information which was either worthless or false; the remaining five per cent provided sufficient accurate reports to justify the expense of the remainder. The first two clauses of this sentence may have a greater validity than the last.” (The last observation was perhaps a tacit hint of the XX Operation.)  Agent Richter may have been sent in to verify whether TATE had been turned, but the fact that the Abwehr never learned anything from Richter did not deter them. The Abwehr no doubt had it confirmed for them how difficult it was to infiltrate an island nation. MI5, even at that time, took pains to ensure that manipulated transmissions took place in locations where the spy was supposed to be, but the state of the technology on the German side at that time was probably inferior to that of the British: even with appropriate triangulation, transmitters could not be ‘pinpointed’ to much less than a circle of 20-mile radius, and there is no evidence that the Germans bothered. Yet the awareness of RDF as a technique for counter-espionage would have registered with them, and would come sharply into focus a few months later.

As a coda, and a point to be picked up later, the British apparently recognized, after the war, the Germans’ superior techniques in detection and direction-finding. In his 2011 memoir of his days at Bletchley Park, Secret Days, Asa Briggs writes that GCHQ acquired a field north of Bletchley that was later named Furzton. “A radio direction finding system developed by the Germans was installed there. Judged superior to all existing British systems, it consisted of an outer circle of forty and an inner circle of thirty smaller metal masts,” he adds. Yet a search on ‘Furzton’ fails to come up with anything else. (Google led me to Hinsley’s and Tripp’s Codebreakers, a book I own, but with no incidence of ‘Furzton’, which does not appear in the Index.) To learn more, perhaps, we must wait for the Official History of GCHQ to appear next year. The overarching conclusion must be that, after the initial excitement in setting up W Division in MI5 in August to track illicit wireless, the transfer of RSS to SIS, and the establishment of the XX Operation, accompanied by the belief that all German agents had been turned, incarcerated or executed, concern about  illicit radio transmissions, whether they came from foreign embassies, maverick civilians, Soviet spies, or even undetected German infiltrators, the demand for prosecution of such activity through urgent and efficient location-finding went somewhat off the boil.

The Funkabwehr

The Nazis had their equivalent of Britain’s Radio Security Service, the Funkabwehr, sometimes translated as the Radio Defence Corps. Yet the Germans came rather later to recognize that the threat of domestic illicit wireless communications required a more committed function. Created by Hans Kopp in 1940, the Funkabwehr reported to the OKW, the Oberkommando der Wehrmacht, and readers may find references to the OKW/WNV/FU, a typically precise but wordy example of how the Germans described their units, Wehrmacht Nachrichten Verbindungen Funküberwachung, loosely the surveillance of radio intelligence and communications. Unfortunately, a good history of the Funkawehr remains to be written, as German records are unavailable. For a detailed history of the organisation, the Wikipedia entry at https://en.wikipedia.org/wiki/Funkabwehr is reasonably solid, but has a very shaky chronology, is written too much in the passive voice, and in my judgment contains several errors. * Moreover, it is highly dependent on a 1946 report compiled by the RSS itself, which can be seen at https://drive.google.com/drive/folders/0B_oIJbGCCNYeMGUxNzk0NWQtNzNhZi00YWVjLWI1NmItMzc2YWZiZGNjNjQ5, a folder in Christos T.’s excellent website dedicated to military intelligence matters. While this account lacks the benefit of historical distancing, and integration of much new material, I shall not repeat here the detailed evolution of the Funkabwehr’s capabilities.

[* The danger of referring to Wikipedia, or indeed any on-line source, is that the entry may change suddenly, or even disappear. The Wikipedia entry on the Funkabwehr has been expanded considerably since I started this article.]

Germany and Great Britain had long maintained ‘Y’ (signals interception) capabilities, the focus of which had been primarily diplomatic and political communications of foreign powers, but assumed interest in military plans and operations as war approached. Britain had listening posts throughout the empire, and Germany had established a similar network within the German borders. The Nazi interest in the years before the war appears to have been directed more against the Soviet Union: by 1937, from their intercept stations at Treunbritzen, Jüterbog, Königsberg and Breslau, they were picking up a large amount of NKVD traffic stretching from Murmansk to Odessa. This activity no doubt continued during the period of the Nazi-Soviet pact (August 1939to June 1941), and helped Hitler prepare for operation Barbarossa.

German Communications (from RSS report)

Yet, as the awareness of possible clandestine wireless activity within each nation’s borders increased, approaches to the problem started to diverge. True, the general methodology and use of technology were very similar, but the geographical and political constrains led the adversaries down different paths. First, the borders in the European theatre of operations remained stable for the British: the Germans had to deal with their fast expanding occupation of new territory. While it provided for a steady increase in suitable locations for interception stations (e.g. Brest, in France), it also increased the possible quantity of subversive communications. It also put more strain on inter-unit communications, since secure landlines were no longer available, and thus exposed more secret information transfer to interception itself. Moreover, the operations were frequently taking place in environments hostile to the invaders, with the risk of sabotage, and, certainly, non-cooperation.

Another aspect was duplication of effort. It sometimes comes a surprise to learn how fragmented the approach of a totalitarian nation could be to intelligence matters. Hitler encouraged rivalries, however, and there was a large absence of trust between organisations. In fact, the function of the Funkabwehr was split between the OKW unit and a section of the Ordnungspolizei (or Orpo) called the Funkabwehrdienst, which was under the control of Heinrich Himmler. Both units were responsible for the location and apprehension of those transmitting illicitly, but for most of the war their missions were divided by what could seem to be an absurd and unproductive distinction. Orpo was responsible for identifying clandestine operations against the government and the regime, while the WNV/FU directed its efforts against activities against the state. How they could confidently conclude which category a transmission belonged to before analysis, or why they discounted the fact that some factions might effectively be fighting both, has not been explained. Britain, on the other hand, maintained a unified control over interception, and generally benefitted from the large amount of trust that existed between the military, the political, the interception and the cryptographic organisations. It was not until 1943 that the Orpo and the WNV divided their tasks more sensibly along geographic lines.

One critical matter that the RSS report brings to the surface is that of distortion of signals, and how the proximity of electrically conductive objects of dimensions close to the length of the wave could affect both reception and interception. What the receivers of transmissions initiated from agents in enemy territory were interested in was content, and weakening of the signal would affect successful reception. Communication was one-to-one: the receiving station would be the sole unit dedicated to trying to capture a transmission. Distortion could mean that the signal was lost completely, or fell into the skip zone. Location was not important to such receivers: indeed, transmitters were encouraged to move around (with those clumsy antennas – but not too far afield so as to jeopardise the signals plan) to evade detection. Interceptors, on the other hand, were rarely interested in content: they probably did not have the resources or time to decrypt the messages. What drove them was location, so that they could quickly eliminate (or turn) the offending agent and equipment. Distortion might not mean complete loss, as multiple detectors had to be in place to perform the triangulation necessary, but it could mean that a faulty indication of location was reached.  

Yet it was all a hazardous business. The presence of interfering objects (buildings, mountains), by radiating signals in new directions, can confuse the process of triangulation, or cause the assumed location to be challengingly large. This distortion can also occur simply because of the erratic behavior of the ionosphere, especially at time of sunrise and sunset. Guy Liddell reported, on February 10, 1941 that ‘the alleged parachutist’s [JAKOBS’s] transmitter from this country was heard again on Sunday but turned out to be a communication between Paris and Cracow’. In a 1944 report, written by British Intelligence to prepare its officers for the invasion of Europe, appears the following observation: “The skip distance of any transmitter is calculable in normal circumstances; but, occasionally, owing to temporary changes in the atmosphere freak results may be obtained, as in the summer of last year when the short wave transmissions of Chicago police cars were clearly (and tiresomely) audible on the south coast of England.” (I am confident that this pamphlet, available at Kew at WO-279-499, was written by Hugh Trevor-Roper: he was the Abwehr expert, and the prose has a donnish flair, and is regularly sprinkled with Latin phrases.) We should also remember that Britain’s scheme of catching all groundwaves by the dispersion of interceptors throughout the country could not conceivably be mirrored in Germany, let alone in its expanded territories. The dynamics of the cat-and-mouse game played between spies and enforcers must be evaluated in this context.

Overall, therefore, the reputation of German counter-intelligence as a ruthless and efficient machine, which has been encouraged by war-movies, and even historians of SOE, is certainly overstated. The Funkabwehr suffered from duplication, tensions of centralisation and decentralisation, inadequate training, poor communications, a shortage of qualified amateurs (unlike Britain’s Voluntary Interceptors), too rapid job movement, insufficient mobile units, sometimes poor quality equipment, and lack of appropriate language skills. Coordinates provided by remote RDF were frequently too vague to ensure successful local house-hunting. Certainly the discovery of the Soviet Rote Kapelle spy network in the summer of 1941 moved operations into a higher gear, but the organisation in France (for instance) remained weak until as late as 1943. The RSS report assesses the technical resources at the outbreak of the war as being ‘completely insufficient’, given the rapidly occurring military victories and the increase in occupied territory’. It tells a story of frequent failure, that it took weeks or even months before a transmitter was at all precisely located. Yet the RSS seemed also to be under the impression that the number of Allied W/T agents was rapidly growing in 1940, an illusion that is undermined by the histories of SOE that have appeared. The more innovative technologies and approaches of the Funkabwehr thus occur well after the period under the microscope in this chapter, and will be analysed in a future episode.

SOE and Wireless: 1940-1942

The SIS organisation in Europe had been greatly weakened by the beginning of war, and the Venlo incident on November 9, 1939 (whereby the Abwehr captured SIS officers in Holland, and gained detailed information about the service’s structures and personnel) crushed it. SOE was launched, with a charter written by the dying conservative Neville Chamberlain, and under the ministerial direction of the socialist Hugh Dalton, in July 1940. Its mission was to perform subversion and sabotage in those countries of Europe controlled by the Nazis. While Chamberlain declared that its operations should be tightly woven in to the greater military strategy of the war, this facet of its decision-making was never really clear. Was it supposed to disrupt the Germans’ efforts to produce war material? Was it designed to initiate minor diversionary attacks that would draw a high degree of military and police resources away from other arenas? Or was it intended to help prepare for the eventual invasion by softening up targets, and impeding troop movements? All these goals were troubled by the fear of what reprisals the Nazis might take on such incendiary activity, and what effect that might have on local morale. Moreover, SOE was always competing for resources – especially for aeroplanes and wireless equipment – and those often unfulfilled demands, hampered by other departments that questioned SOE’s effectiveness, meant that SOE had a very chequered history in the first two years of its existence.

The sources on SOE are fragmented. M. R. D. Foot’s SOE in France, originally written in 1966, and reissued in 2004, is an ‘official’ history, part of the Government Official History Series, but, as is clear from its title, covers France only. (In an interesting sidenote, Foot himself, in his 1976 work, Resistance, refers to SOE in France as a ‘quasi-official’ history.) Foot wrote another volume covering all of SOE, SOE: The Special Operations Executive 1940-1946, in 1984, but it is not an ‘official’ or even ‘authorised’ history. Its chronology is hazy, and it provides little detail on wireless equipment and procedures. After the war, an internal history was commissioned from an Oxford don, W. J. M. Mackenzie (who had not been employed by SOE), and was eventually published, in 2000, as The Secret History of SOE: The Special Operations Executive 1940-1945. In all three books, the coverage of wireless is very sketchy until 1943, after SOE’s own research and manufacturing facilities had been set up, and Colonel Gubbins rather belatedly introduced more rigorous signals procedures. Various memoirs refer to the use of wireless, but they are not always reliable.  A number of files have been released to the National Archives in recent years, but few records of SOE’s activities in the early years appear to have survived fire, destruction or the weeders, and what have endured are (so far as I can judge) all undigitised

This report focusses on SOE in France, as it was the earliest field of operation, and it is here that the most pressing lessons of wireless usage were learned. SOE had two units working in France: the F Section, which was run as a British operation, and the RF section, which was a Gaullist unit for which French nationals only could work. F thus depended mainly on agents of Anglo-French nationality who spoke the language fluently.  And it took many months before SOE sorted out is mission, recruited and trained people, overcame political opposition, and were able to start placing agents deep inside France. It had infiltrated a few agents equipped with wireless by sea, but their communications were apparently spotty. The first confirmed F agent to be parachuted in with a wireless set was Georges Bégué (aka George Noble), who arrived in unoccupied central France on the night of 5/6 May 1941.

It might be expected that the local populace would be more supportive of parachutists sent in to hinder and harass the invader, but it was not necessarily so. Up until Barbarossa, the French communist party had welcomed the Nazi allies of Moscow, and rapidly had to change their stance after June 1941. Before then, however, communists were a threat to subversive activities as possible informers. Even in Vichy France, considered to be safer territory, many peasants were loyal to the administration, and would betray illicit movements to the authorities, and hence to the Germans. SOE’s policy with wireless operators was open to criticism: it would send in a team of three (agent, courier, and wireless operator) rather than devolving the task of transmission and receiving to the agent him- or her-self.  Frequently the operator spoke no French, and might be idle for weeks at a time, which meant concealment and exposure were a constant concern. Yet progress was slow. Lorain (see below) writes that there were only two clandestine stations working in France for Section F in May 1941, and a year later, still only seven.

Thus one has to treat Foot’s claims about the rapidity with which the Germans developed direction-finding techniques with some skepticism. He reports that ‘the German wireless interception service had detected Bégué’s transmissions almost at once, had begun to jam them within half a week.’ The Vichy police was involved, and ‘D/F vans joined in the search’. Elsewhere, in a general commentary, Foot writes: “The German intelligence service’s wireless direction-finding (D/F) teams were numerous and efficient, probably better than the British, for whom Langelaan [George Langelaan, Knights of the Floating Silk, p 220] claimed that if ever an unidentified transmitter was heard ‘in a manner of minutes a first, rough direction-finding operation had been accomplished.’” Again citing Langelaan, Foot then goes on to make the following rather nonsensical observation: “If the transmitter was anywhere in the United Kingdom, in less than an hour experts equipped with mobile listening and measuring instruments were converging on the region where it had been located.” Why an official historian like Foot would rely on Langelaan as a source, when the author was an SOE agent who probably received the information second- or third-hand, is not clear. (Admittedly, Foot would not have been able to find reliable information in the archives, but that is no excuse for such slipshod reporting.) From other accounts (such as Liddell’s Diaries), it is quite clear that, during this period, the approach by RSS to suspicious signals was much less rigorous.

As for what the capabilities of the Nazi teams were, ‘converging’ might mean location-finding rather than physical movement, but the proximity of Augsburg and Nuremberg to each other [see below] would mean any attempt at triangulation with Brest on sites in Britain would be a very haphazard, as well as pointless, exercise.  Nevertheless, Foot goes on to write: “French operators in the field early discovered that a long transmission in a large town would probably bring a detection van to the door within thirty minutes. The Germans soon worked out a technique for establishing what part of a town a clandestine operator was working in, by cutting off the current sub-district and noting when the clandestine transmission was interrupted; then they would concentrate their efforts on the sub-district affected, and hope to track down quickly at least the block, if not the building, the set was working from.”

In his general book about SOE, Foot reinforces the message. “In towns, sensible organisers and wireless operators took care not to see too much of each other; for the wireless operator was always the circuit’s weakest point. The Germans, like the British, kept a constant watch on every wireless wavelength, and it took only twenty or thirty minutes for a team of their armed direction-finders to get within a few yards of an operator who was fool enough to remain on the air so long. Relays of thirty clerks with cathode-ray tubes in the Gestapo’s headquarters in the Avenue Foch in Paris, for example, kept up a continuous watch on every conceivable frequency. When a new set opened up, it was bound to show up on a tube; the frequency could be read off at once. In a couple of minutes, alerted by telephone, direction-finders at Brest, Augsburg and Nuremberg were starting to take cross-bearings; within a quarter of an hour, detector vans would be closing in on the triangle a few miles across that the cross-bearings had indicated. Some of SOE’s early organisers in France and Belgium insisted on sending messages so verbose that their operators had to remain at their morse keys for hours at a time; and, inevitably, they were caught.

German Position-Finding, Phase 1 (1942?) (reproduced from Pierre Lorain’s ‘Secret Warfare’)

It did not take long for Gubbins, as head of operations, to spot what was wrong, or for the signals training school at Thame Park to start to impress on operators – as Beaulieu explained to organisers – that mortal danger lay in trying to send long messages by wireless.”

Yet all this is undated, and perhaps an indication why this analyst is wary is that Foot immediately follows this last passage with the following: “By the winter of 1943-4 – hardly before time – there was an order: no wireless telegraphy (W/T) transmission was to last longer than five minutes.” In the context of the war, this is an enormous chronological jump. Foot lists several other operations (Forman and Labit, DASTARD, Bloch) in the second half of 1941 that he claims were terminated because the operators stayed on the air too long, and were trapped by the efficiency of German detection-finding. Yet it is perhaps more likely that many of these agents were betrayed by sloppy tradecraft, or visible behavior that prompted the interest of citizens who felt it their duty to report such activity before they were arrested for ignoring it. In fact Mackenzie tells us that Labit (the wireless operator) had to escape to the Unoccupied Zone without his set, while his partner Cartigny was probably shot. Some gave the game away by weak identity cards, or obviously wrong serial numbers on notes, the same types of error that had bedevilled the LENA spies. In Resistance, Foot undermines his argument by writing: “Early in the war, the Germans worked the process [of interception] clumsily, but by the spring of 1943 they had main intercepting stations in Augsburg, Berlin, Brest, Nuremberg, and no doubt elsewhere.” Again, a distressing lack of precision, and a big chronological leap.

In his largely pictorial study of the use of wireless in the French Resistance, The Clandestine Radio Operators in France (2011), Jean-Louis Perquin presents an arresting account of the German special unit ‘dedicated to the detection of clandestine emissions’, describing a complex web connected to three detection-finding centres located in Brest, Augsburg and Nuremberg, and backed up goniometer trucks with equipped with the latest technology. Yet, again, chronology is vague: the text indicates that the procedure described was deployed in 1943. There is no evidence of the state-of-the-art in 1941. Perquin explains that RF agents were trained by British instructors, and also dependent on SOE equipment. “In Autumn 1941”, he writes, “following the numerous loss (sic) suffered by those specialists and considering how such losses were threatening the very existence of the networks, the SOE decided to create a security course in Grendon, Buckinghamshire.” Yet, if losses of agents were due to overlong transmission times, or failure to switch frequencies, one might think the problem could have been swiftly addressed through tighter discipline. Gubbins’s edict of winter 1943-44, after ‘it did not take him long’ to work out what was happening, simply seems absurd.

It appears that Foot and Perquin were using the same source, but it is not clear what it is. In Resistance, Foot declares his heavy reliance on Pierre Lorain’s Armement Clandestin (1972), a book that also appears in Perquin’s Bibliography, which was translated and published in English as Secret Warfare in 1983. Lorain gives a much more reasonable account of what happened, and it is worth quoting three paragraphs in full.

“German detection methods had made decisive progress in 2 years. In 1941 and 1942, the localization of a clandestine station was extremely difficult. It could be carried out only if the operator transmitted on the same days of the week, from the same site, and on the same frequency during several consecutive hours. Direction-finding operations were not yet automatic, and panoramic reception was non-existent. The scanning of all usable frequencies was necessarily very slow and left substantial gaps.

In addition, during the final approach, each Gestapo agent had to hide a heavy suitcase containing a receiver with a loop aerial under his coat. A Tirolean cap or Basque beret tilting down over his ear just barely hid an earphone. Their general posture aroused the curiosity of even the most naïve of passersby.

The arrest of a radio operator thus required long months of continual surveillance, the operation was complicated by the fact that if a clandestine operator was spotted in the unoccupied zone of France (controlled by Vichy), the Germans could only signal the suspect frequency to the French radio control group at Hauterive near Vichy. The latter promised to look into the matter, but secretly warned the clandestine station to move as quickly as possible, and then supplied the Germans with an almost completely false position.”

The Funkabwehr article I referred to before contains nothing about operations in France against SOE. I have been advised that the unit’s records reside somewhere in Moscow, so one cannot judge how much of Lorain’s account is true. Yet it seems as if Foot’s official history tries to deflect attention away from other systemic problems in SOE’s deployment of wireless. (His comments above need to be transferred en bloc to the state of the game in 1943 onwards, a period I shall cover in a later article.) A careful reading of Mackenzie would suggest that a number of severe problems affected both the F and R/F operations in France until 1942: a lack of radio expertise for establishing reliable wavelengths and schedules, leading to failed use; struggles with transporting and concealing the heavy equipment; inappropriate choices of agents who had unsuitable personalities; careless practices by the wireless operators, who were not always trained properly; inappropriate centralisation of transmissions because of shortage of equipment, leading to intense and long broadcasts; betrayal by agents (such as the notorious VICTOIRE); the unreliability of the local police in Vichy France. It was easier for SOE to blame German direction-finding.

And it seems more probable that other territories – and another enemy – were the arena in which the Reichssicherheitshauptamt improved its detection capabilities. As I shall explore, the Funkabwehr was provoked into quick reaction after Barbarossa (June 1941), as the Red Orchestra started tuning up, primarily in Northern France and Belgium. Colonel Buckmaster, who headed F Section, reported that, as late as August 1942, in the Occupied Zone, he had only two wireless sets, of which one was operational, while in the Unoccupied Zone, the numbers were six and four. In Belgium, however, the following distressing tale emerges, as German counter-action took place. In the First Quarter of 1941, two out of 9 sets had been captured and operated by the Germans: the figures for the next three quarters were 5 out of 6; 8 out of 8; and 7 out of 8. I shall return to the topic of whether German RDF advanced faster in Germany, because of the activation of the Red Orchestra after Barbarossa, and explore how soon operations in France were able to take advantage of such breakthroughs. Overall, my conclusion would be that the sluggishness with which SOE mobilised its wireless communications, and the slow but steady steps by which the Funkabwehr moved into action against Communist spies in the latter half of 1941, suggests that Foot’s suggestions of hyperactive German detection-finding in 1941 are premature, and that the losses were due to other causes.

In any case we know that SOE was inhibited by the fact that SIS controlled its cyphers and communications until June 1942. Up until then, it had had to accept whatever equipment SIS gave it – clumsy and heavy apparatus. As Foot writes: “Agents were not best pleased at SIS’s first offering, a plywood box that weighed some 45 lb. (20kg), already looked old-fashioned and contained a Mark XV two-valve transmitter fitted with a morse key, and its power-pack, a 6-volt car battery.” Foot does not describe the travails that agents lugging a 45-lb. suitcase around an unfamiliar terrain must have experienced, let alone the difficulties in setting up a suitable aerial without drawing attention to themselves.

The conclusion about SOE’s (and specifically Gubbins’s) track-record concerning wireless up to 1942 must be that the operation was needlessly clumsy. It cannot all be blamed on SIS.  I read A. R. B. Linderman’s Rediscovering Irregular Warfare: Colin Gubbins and the Origins of Special Operations Executive (2016) in the hope of acquiring some deeper insights. Linderman informs us that a Frederick Nicholls served under Gubbins as director of signals during World War II, but that is the only mention that Nicholls merits in the Index, and the story is disappointingly thin on wireless matters. Maybe the skills of Nicholls, who ‘had managed to establish wireless communications with the British Embassy in Kabul during the Third Anglo-Afghan War’ (which occurred between May and August 1919) were stretched by the exigencies of communications in Nazi-occupied Europe if that was his premier achievement. The clumsiness of SOE’s wireless strategy would however endure until the end of the war, as I shall explain in a later episode.

Major-General Sir Colin Gubbins

The Red Orchestra

While the Comintern and its allies had enjoyed successful experiences with illicit wireless transmission in the 1930s, Stalin’s purges of 1937 and 1938 had required much of the Soviet Union’s networks in the West to be rebuilt. It was not hard to find native Soviet sympathisers outside Germany, since the propaganda of communism as the only effective bulwark against fascism had worked effectively both on the disenchanted ‘toiling masses’ as well as on the guilt-ridden intellectuals. Since Hitler had either executed, incarcerated or forced into exile any members of the Party, or outspoken supporters of communist doctrine, Germany remained a more difficult country to penetrate. But neighbouring nations provided a rich source of potential spies and informants: many eastern Europeans found homes in the Low Countries and France, for instance, and were able to fade into the background without being conspicuous. Britain had its own nests of spies, of course, both from the older universities – who had successfully detached themselves from any association with the Communist Party of Great Britain – as well as more traditional working-class enthusiasts. But these eager adherents to the cause of the proletariat needed managing, and directing in their efforts. They needed intermediaries, and they need a mechanism for getting the fruits of their espionage back to Moscow.

Soviet espionage had three arms – the Comintern, the NKVD, and military intelligence, the GRU. David Dallin, in his epic Soviet Espionage (1955), informs us that, as early as late 1935, “Only a comparatively small Soviet apparat now remained in Germany: the greater part of the network had either been dissolved or moved abroad. The OMS had moved with the Comintern’s West European Bureau, the WED, to Copenhagen; the passport apparat had gone to the Saar, and Soviet military intelligence to Holland and France; the party leadership had migrated part to Prague and part to Paris.” Thus what survived the purges (with the GRU the most hard-hit) was still a very fragmented approach to intelligence-gathering, with no guarantee that it would be efficiently shared back in Moscow. In Volume 2 of his biography of Joseph Stalin, Waiting for Hitler, 1929-1941, Stephen Kotkin writes (p 496) that a dozen NKVD station chiefs abroad were arrested in 1937-1938, and that, in Berlin, ‘Stalin cleaned house, arresting nearly every NKVD operative there’. The GRU suffered even more, with 182 operational staff arrested in the same time-period. Yet the growing menace of Germany and Japan meant that, under Beria, a rapid repopulation of the networks had to be accomplished.

The International Brigades in Spain had constituted a useful source of potential operatives, as well as an opportunity to grant new identified to infiltrated agents, by virtue of the passports that had been stolen from Brigade members when they entered Spain. Alexander Foote was a famous example of such a footsoldier who was plucked from obscurity to be sent to Switzerland to received training in wireless operation from Ursula Kuczynski, agent SONIA. At the end of 1938, agents in their dozens started arriving in Europe, as well as the Far East and the United States. Like the Nazis, but with far more deliberation and craft, the Soviets chose, or allocated citizenship to, agents who would never arouse suspicion owing to domestic (Russian) nationality. The complex borderlands of the old Russian Empire provided a rich environment for muddled heritage and absence of reliable documentation, in order to allow unverifiable accounts of life-history to be passed off.

Accounts of training for wireless activity are thin on the ground. SONIA’s memoir (which in these technical aspects is probably much more reliable than in political observations, such as her absurd accusations of imperialistic infiltration helping to crumble the Soviet Union) is certainly not typical.  For she was respected enough to avoid the purges, and also had had a long experience in China as a wireless operator before being recalled to Moscow for leave and ‘discussions’ in late 1935. Her account is unfortunately very muddled in chronology, but it is educational in that it clearly identifies some of the problems that illegal wireless operators would experience anywhere in Europe. After a brief interlude with her family in London, she was then sent to Danzig, then a ‘Free City’, where she was instructed to ‘obtain residence permits, find work to legalise our existence, and set up our transmitter for radio contact with the Soviet Union’.

SONIA had been instructed how to build a transmitter in China, by her lover, Ernst, and claims that she received a response from Moscow immediately she set up her apparatus. Her task was to advise a group of labourers undertaking occasional sabotage at a shipyard building U-Boats in Danzig (where the Nazis were outrageously breaching the constitution that the city had been granted), and transmit on their behalf. At one stage, she and Rolf moved to a new house, but discovered that proximity to a power-station made signals inaudible, and she had to take her equipment to an apartment – a lesson that probably stood her in good stead later in England. Yet she immediately stumbled dangerously: the apartment block she chose was the residence of several Nazis, and one day the wife of them asked her whether the reception on her radio had been affected by interference. Her husband had told her he believed that someone was transmitting secretly, and was going to arrange for the block to be surrounded. SONIA even mentions triangulation of radio detection, which would have been a very early indication of the Nazis’ fears – and progress in allaying them.

Soviet ‘Sever’ Wireless Model

SONIA did not take the right steps, however. She broadcast again, from the same apartment at the same time, instead of the middle of the night when neighbouring radios would not have been on. She should have moved to a friend’s apartment, or returned to Warsaw. It appears that she was in awe of doing anything without Moscow’s approval: the outcome was that she was ordered to return to Poland as she could no longer transmit. Thus, when she met her boss, Comrade Andrey, in Warsaw, she asked to receive further training in wireless construction and use in Moscow. That need was reinforced by her receiving a severe electric shock one night, burning her hand. SONIA would pay two visits to Moscow during 1937 and 1938 (she admits that the details of each congealed into a blur). Her return to Poland was uneventful. She had to return to Danzig to help a comrade set up his transmitter, and admits that he was ‘slow on the uptake’, so maybe Moscow’s selection and approval processes for its agents were not very rigorous. Communist fervor may have been considered more important than intelligence and the right psychological profile. SONIA felt she was not accomplishing much: “The Danzig people had their own radio operator, the Bulgarian comrade produced little information. I only transmitted once a fortnight.”

In August 1938, it was decided to send her to Switzerland, where the plan was to infiltrate agents into Germany, to make contacts at the Dornier aeroplane factory in Friedrichshafen. And that is where the story of ‘Sonia’s Radio’ picks up, with her eventual successful establishment in Britain in the spring of 1941, and her activation as a wireless agent a few months later. She met up with Sándor Radó, who as agent DORA had been appointed head of the Swiss network, but had no wireless skills. In his memoir, Radó writes how Sonia visited in him in December 1939, and how the following month his radio contact with Moscow had been established. He also describes a visit in March 1940, set up by Moscow Central, by someone he knew only as KENT (see below). KENT spoke authoritatively about the necessity of secure wireless procedures, stressing the importance of changing the number and times of transmissions as often as possible ‘as the best protection against being located’. He added that operators should move around different residencies, as well. “Keep changing them if you can – but again, avoiding any kind of system. The thicker the fog, the better.” It suggests, again, that a prematurely intense fear of radio-detection capabilities existed with the Soviets, and that their listeners back in Moscow would be prepared to listen around-the-clock for their agents’ transmissions. But it was easier to preach such practices than to follow them.

The Soviet defector Walter Krivitsky also gave hints of subversive radio activity in Central Europe. In his memoir In Stalin’s Secret Service, he related how Marguerite Browder, the sister of the head of the US Communist Party, Earl Browder, had graduated from the school in Moscow that specialised in wireless competency, and had then been sent abroad as an illegal with an American passport issued in the name of Jean Montgomery. “During 1936-1937 she worked in Central Europe where she laid the ground for the establishment of our secret radio station,” he added, with an unhelpful lack of precision. If we can rely on Krivitsky, shortly before his recall to Moscow Sergei Spiegelglass, sent on a deathly mission by his OGPU boss Yezhov, tried to get Krivitsky to assist in the assassination of his friend and colleague Ignace Reiss. When Krivitsky demurred, he then asked Krivitsky to hand Browder over to him, as he had an ‘important job’ for her in France. The implication in Krivitsky’s rather fractured account is that he managed to warn Browder of what Spiegelglass had in mind for her, and that she was able to continue with her wireless activities.

In his biography of Kitty Harris, The Spy With Seventeen Names, Igor Damaskin informs us that the European network was issued with much more sophisticated wireless equipment at the end of 1936. Kitty Harris, who was Marguerite Browder’s sister-in-law, was brought back to Moscow for retraining in January 1937. She apparently showed little aptitude, and it was determined that ‘any more technical training would be a waste of time. She was later assigned to be Donald Maclean’s handler in London and Paris, where she specialised in photography.

Yet wireless usage in broader Europe at this time was sparse. It was not necessary. Moscow had its eye on the long term. The presence of Soviet legations or embassies in most capitals of the West provided a mechanism for information to be collected and then sent by diplomatic bag or courier back to Moscow. As a long-term measure, a wireless centre was set up in Brussels, where Trepper, as the new leader of the western organisation, replacing Walter Krivitsky, installed himself in March 1939. Yet, as Heinz Höhne tells us in Codeword Direktor, Trepper left it dormant, concentrating first on recruiting a team of informers, and enlarging his contacts with the world of business, the military and diplomacy. Even when war broke out, there was no quick change of operation. Only when Nazi Germany started its invasion of Belgium and the Netherlands in May 1940 did hasty adjustments have to be made. Even though the Soviet Union was in a non-aggression pact with Germany, its needs for information on Germany’s plans, and the reactions of France and Great Britain to Nazi movements, placed increasing pressure on Trepper and his cohorts to deliver.

Communication switched to radio sets when the Germans occupied Brussels, and the staff of the Soviet legation was withdrawn. In August, 1940, Trepper moved with his mistress to Paris, leaving there the unreliable playboy Sukolov-Gurevich, known as KENT, as the only agent capable of representing the GRU network. The Sokols were then recruited as wireless operators by the Soviet Embassy, and trained by someone called Duval. By June 1941, the Soviet Military Attaché, Susloparov, had moved to unoccupied France, and Trepper was in Vichy on the day that Germany attacked the Soviet Union. Meanwhile, in Berlin, more urgent plans were made in April 1941 to establish direct radio contact between the cells led by Arvid Harnack and Harro Schulze-Boysen, the Soviet spies in the heart of the Nazi administration. (Even if Stalin did not believe the rumours of a Nazi invasion, some of his intelligence officers were presumably more realistic.) In late May, two transmitters were sent by diplomatic bag from Moscow to Berlin, ‘one a small battery model and the other a large mains-powered set portable enough when broken down to fit in a suitcase’, as Costello and Tsarev describe. Harnack was chosen to be the operator, but declined, delegating it eventually to an engineer named Behrens, while Schulze-Boysen took up the challenge for his group, with much more eagerness, selecting a factory technician called Hans Coppi.

Costello and Tsarev report further: “The Berlin groups had established several safe locations on the upper floors of trustworthy colleagues’ houses in the countryside outside the city where the transmitters could be assembled and their aerials run up into the attics in order to communicate with Moscow. The Centre arranged to keep a listening watch on set hours and days of the month, which were multiples of the numbers four and seven.” Coppi received training from the local NKVD office, and successful transmissions were made in the beginning of June, and picked up and decrypted in Moscow. The infrastructure was in place when Operation Barbarossa was started. As Dallin records the situation: “This, then, was the setup on the eve of the Soviet-German war: a number of espionage agencies with radio facilities and sources of information, organized but dormant, in Belgium and Holland; rudimentary apparats in France and Denmark; a few trading firms established as covers in Brussels, Paris, and Geneva; a promising start in Switzerland; and a group of enthusiastic but inexpert operators in the German capital.”

Summary

Thus, as the wartime alliances solidified in the summer of 1941 (with the USA to join the Allies a few months later) mainland Europe entered its most intense couple of years of illicit wireless transmission and detection. Many agents – as well as dedicated wireless operators – did not have a suitable profile for the tasks at hand, and had been sketchily trained. The equipment they used was frequently clumsy and unreliable. The support structures behind them had not always analysed the variables of distance, sunspots, terrain, or mechanical interference in depth enough to define the wavelengths and times that they should best operate. They frequently disobeyed best practices in their transmission techniques, and ignored rules of basic spycraft. But they all probably had an exaggerated sense of the state-of-the-art of enemy detection and direction-finding techniques at the time, and how efficient it was, and certainly used such capabilities as an excuse for sloppy behaviour when agents were apprehended. All this would change very rapidly as the battle of wits intensified in the second half of 1941, when Nazi Germany honed its capabilities in the face of the Rote Kapelle activity. The major significant conclusion is that, as Germany intensified its capabilities for detecting the threat of domestic (or imperial) illicit wireless, Britain moderated its own home coverage. Through policy and organisational change, it concentrated much more on transmissions in mainland Europe, and on the interception and decipherment of official transmissions made by the Nazi war machine.

The final observation to be made is to note the anomalous attitude of British Intelligence towards its Nazi enemy during this period. While crediting an exaggerated efficiency and skill to the Abwehr’s counter-espionage activities, in the form of effective Radio Detection- and Location-Finding, it attributed the obvious ill-preparedness of the agents (training, language, identification papers, etc.) it sent to Britain to the stupidity and clumsiness of the same organisation. Yet, while priding itself on its superiority in both regards, the British intelligence services (in this case MI5, RSS & SOE) developed casual habits in its interception of domestic illicit wireless, and also sent agents to the continent who were likewise unready or unsuitable for the challenges of working in hostile territory.

(I am again grateful to Dr. Brian Austin for giving me guidance on matters of wireless technology. Any mistakes or misrepresentation are mine alone.)

Sources, and for further reading:

SOE in France by M. R. D. Foot

SOE, the Special Operations Executive by M. R. D. Foot

The Secret History of SOE by William Mackenzie

Resistance by M. R. D. Foot

Deceiving Hitler by Terry Crowdy

Soviet Espionage by David Dallin

Codeword Direktor by Heinz Höhne

Unternehmen Seelöwe by Monika Siedentopf

Rediscovering Irregular Warfare: Colin Gubbins and the Origins of Special Operations Executive by A. R. B. Linderman

Secret Warfare by Pierre Lorain

The Clandestine Radio Operators by Jean-Louis Perquin

Wireless for the Warrior, Volume 4 Clandestine Radio by Louis Melstee and Rudolf F. Staritz

The Third Reich is Listening by Christian Jennings

SNOW: The Double Life of a World War Spy by Nigel West & Madoc Roberts

Operation Blunderhead by David Gordon Kirby

Sonia’s Report by Ursula Hamburger

Codename Dora by Sándor Radó

The Duel by John Lukacs

Double-Cross by Ben Macintyre

Hitler’s Spies by David Kahn

Fighting to Lose by John Bryden

Deadly Illusions by John Costello and Oleg Tsarev

Secrets of the British Secret Service by E. H. Cookridge

Codebreakers: The Inside Story of Bletchley Park by Alan Stripp & Harry Hinsley

Bodyguard of Lies by Anthony Cave-Brown

Secret Days by Asa Briggs

The Searchers by Kenneth Macksey

The Spy With Seventeen Names by Igor Damaskin

In Stalin’s Secret Service by Walter Krivitsky

The Guy Liddell Diaries, edited by Nigel West

The National Archives at Kew, London

This month’s Commonplace entries can be found here.

1 Comment

Filed under Espionage/Intelligence, General History, Politics, Technology

Struggles at the Desktop

Monitoring the home security system at 3835 Members Club Boulevard

[Warning: This article may not be suitable for readers of a sensitive disposition. It describes encounters with information technology that may be disturbing to some.]

“Nowadays if there is an error in the input program the computer not only detects it but gives the approximate description and location of the error and recommends procedure for correction.” (Gerald S. Hawkins, in Stonehenge Decoded, 1965)

When IBM hired me as a trainee Systems Engineer in 1969, it was not because of my data processing skills. That day in late August, when I walked into the Katherine Street office in Croydon, Surrey (shortly before the branch moved into the new building on Cherry Orchard Road), I did not know the difference between a punched-card and a paper-clip. It was not a classical career beginning, and not a carefully-planned strategic move. In an indecisive third year at Oxford, I had applied to take the Certificate of Education after the completion of my degree in Modern Languages, but soon began to have doubts. On a weeklong visit to a local primary school in Purley, Surrey, before the first term started, I had innocently queried the headmaster as to why the classes did not appear to be learning multiplication tables by rote. “Oh, Mr Percy!”, he replied with a condescending smile. ‘We don’t do tables any more!” For this was the era of ‘child-centred’ learning, where every infant had to discover for him- or her-self that 7 x 8 resulted in 56, and so on. I recall the way that tables and mental arithmetic were drilled into my generation about fifteen years earlier, and how the pattern of number combinations has stayed with me ever since. In 1968, however, I was entering the world of Progressive Education.

Perhaps my aspirations were also checked by my term of teaching-practice. Having had a term of almost total inactivity, owing to my being on crutches because of a rugby injury, I was informed, in December 1968, that I was urgently needed as a replacement at Bognor Regis * Comprehensive School, as the previous teacher of Russian had been fired for getting one of his pupils pregnant. I did not learn the cause of the summons until I arrived: the school was also going through a painful merger of a grammar-school with a secondary modern, which also dampened what remained of my enthusiasm. Halfway through this term, I decided that a quick return to the classroom was perhaps not the most life-enhancing prospect to be contemplated. Taking advice from some outfit that suggested that my interest in chess, bridge, crosswords and logic puzzles might open up some doors in the computing industry, I secured interviews with some manufacturers, of which NCR and IBM were the most satisfactory. I took care to complete my Certificate of Education so that I could have a back-up career lest the corridors of business found my talents wanting.

[* Bognor Regis is a coastal town in West Sussex. It gained its regal addendum after King George V recuperated there, and the monarch’s dying words have been apocryphally reported as ‘Bugger Bognor!’. When Ursula Kuczynski (agent SONIA) needed a place for her children to stay while she returned to the Soviet Union in the summer of 1938, she left them with a friend in Felpham, which is part of Bognor. There is no truth to the rumour that I was in 1969 undertaking, under deep cover, some early sleuthing into Sonia’s contacts.]

Unfortunately, IBM was a little slow to snap up the opportunity to make me an offer, so I had to write to them to explain to them that this entrepreneurial youth was thirsting to make his contribution to the computing revolution. Perhaps the company was waiting for such a show of initiative, since I was rewarded with an appointment at the Head Office in Chiswick, to meet one of their Personnel Managers (no ‘Human Resources’ in those days: employees were certainly not ‘associates’, and customers were assuredly not ‘guests’). I was delighted to find that this benevolent soul had also studied Russian at Oxford. He started to quote me a quatrain of Pushkin’s, which I was happily able to complete. I passed the interview. I was in.

Before I started the eight-week basic training course at Sudbury, Middlesex, I had a week in the office, where I was directed to a small room, and given a Programmed Instruction text on IBM’s System/360 to work through. These matters were all rather daunting to me, and I recall I had to interrupt my study to ask the Systems Engineering Manager what the meaning of some concept was. It all comes back to me quite clearly: I wanted to know what was special about the sixteen ‘registers’ of any 360 computer system. Registers were (and no doubt still are) the mechanism by which the locations of computer memory were addressed, but they also seemed to have some properties that lent themselves to high-speed arithmetic. Somewhat confused, I asked the manager whether he could explain their nature to me. “Oh, I never really understood all that stuff”, he said. “I wouldn’t worry about it.” I think we adjourned to the squash court soon after that, and I gave him a good runaround in return.

The Systems Engineering class was tough. All new recruits were required to go through the same basic training, to make sure they were immersed into the IBM way of doing things. I recall a few students who had already served several years with IBM’s rival, ICL, and were thus already very familiar with the concepts and practice of data processing. Most of the graduates straight from university had scientific backgrounds, and had used computers in their laboratory work. There were times when I wondered whether I would make it. My ability to learn seemed to correlate exactly with the ability of the individual instructor to present topics in schemas that matched how my brain was able to integrate new ideas, namely very logically, with clear step-by-step evolution, and no grand jumps that left canyons of unexplored territory behind. Gradually, things began to make sense. I completed the three stages of the training about a year later, and was ready to roll.

Unfortunately, IBM was not sure at that time exactly what the role of systems engineers was, as anti-trust threats had meant that the company could not hand out systems engineering resources to its customers for free. At the same time, we were neophytes eager to learn by practical experience, while the projects we were given were haphazard, not always suitable, and not always educational. I soon learned that I liked coding, appreciated the value of well-designed and well-implemented systems, and became very frustrated with poorly written documentation. And I did have a knack for working out what was at fault when things went wrong, although that experience was marred by a disastrous project where I was asked to make some changes to a Vehicle Scheduling Package for a prominent and demanding customer. There was no guide to how the product worked, and I stumbled for weeks in trying to tweak it to meet the idiosyncratic needs of the customer. I received no help: the project was simply abandoned, I believe. But two lessons started to emerge in my mind: i) the knowledge that there was a logical explanation for every computer failure, and ii) the importance of good diagnostics being built into any product.

I move forward seven or eight years, and two jobs later. I was working as European Customer Service Manager for a small American software company. Our flagship product was known as a transaction-processing monitor, an adjunct to the operating system that handled communications with a network of terminals and managed the user programs that the customer wrote to provide on-line business functions. One of the challenges with this software configuration was that a motley set of technologies all operated in one partition, all clamoring for resources, and all potentially stepping on each other. Much of the code was written in low-level Assembler language, which provided greater manipulative power, and faster execution speeds, but also provided opportunities for corrupting storage occupied by other software. Frequent were the ‘core dumps’ (we still called them such, even though ferrite cores had been superseded as memory components by then) that were mailed in by customers when the system blew up, and we were unable to detect what had happened over the telephone. Then the support team would compare the state of computer memory with source listings of our product, in order to find out where our product (it was frequently the fault of the product) had gone wrong.

One particularly stubborn problem endures in my memory. A prestigious customer had experienced an execution failure, not recreatable, that caused the partition to explode. (The customer was actually the institution where the atom spy Klaus Fuchs, for whom Sonia had acted as courier in 1942-43, was working when he confessed in January 1950: there is no truth in the rumour that I was sent in by MI5, under subterfuge, to undertake an audit of its security procedures.) The requisite hundreds of sheets of print-out were sent in. No one could work out what had happened, and I devoted hours and hours to it. Eventually, I determined that it looked as if an error routine in IBM’s telecommunications package, VTAM, had failed to save properly the register contents that had been passed to it (and which had to be restored when the routine had completed its work), as all processes used those same registers I had been puzzled about back in 1969. I called the customer with my tentative suggestion, and asked him to pursue the matter with IBM. The next day he called back: indeed, one of the error handlers was incorrectly saving and restoring registers. He apologized for not searching for, and applying, the fix that would remedy the problem. Much goodwill was gained.

The second experience that reinforced my earlier lesson was in helping to roll out a new feature in the product, something called ‘Multiple Record Hold’ (MRH). The previous version had allowed only one file record to be held at a time, which was a heavy constraint. If a user application wanted to prevent anyone else accessing a customer record, say, while it then checked an inventory record that it might want to deplete, the systems designer was in a bind. MRH addressed that problem. But our developers designed and coded the feature too quickly and carelessly. Several occasions would arise where the programmer would try to invoke the feature inappropriately (for example, invalid keys, or multiple requests to the same file), or the software detected something illogical. It would return an ‘L’ code to the program, indicating such. But the programmer had no idea what it meant. There must have been several dozen places in the source code where an ‘L’ error code was returned. We, as support personnel, had to trace through the record of programmatic requests, and the source code listed, to detect at what point in the logic the ‘L’ had been returned, and then provide an explanation. But it could all have been made so simple: an auxiliary area existed where a return code could have been posted, and a corresponding piece of documentation could have explained what every code meant, with an enormous benefit in productivity. I was just about to start coding this enhancement when I was invited to work as Director of Technical Services for the parent company in Norwalk, Connecticut. At that time the flagship product was on the way out: the feature was never implemented. And so my wife and I, with ten-month-old son, moved to the USA.

[In parenthesis, for the more technical among my readership, I should also mention here that an unusual feature of this product was that the Control Program was written in a high-level language, COBOL, a decision presumably made in the interests of clarity and maintainability, not in the cause of performance. But when some advanced features were added to the product, it became necessary for the CP [not the Communist Party] to access low-level bitstrings, something COBOL cannot do. Thus an Assembler (low-level) language subroutine called GETBITS was added, to return statuses for further decision-taking and logic-branching. I recall very clearly how one of our most demanding – and shrewdest – customers in the UK, when undergoing performance problems, ascertained, through the use of a testing device, that GETBITS was consuming 6% of all machine cycles on its 370/145 – an enormous amount. Furthermore, when I inspected the new CP code, I discovered that, in many circumstances, the GETBITS routine was being invoked, but the CP was then taking branches that were completely independent of the results of the call! When I vaguely suggested to the President of the Company (who had probably written much of the original code himself) that I could rewrite the whole CP in Assembler language on my weekends, and deliver a much faster system to our customers, he declared, very seriously, that anyone who attempted that would be fired. He still relocated me to report to him in Connecticut, but later gracelessly told me that he only did so because the Director of R & D persuaded him. On such whims do whole lives change.]

The reason for this long introduction is that I recently had to replace my home PC, and experienced massive problems. For some months, my old HP Pavilion had been warning me of its imminent demise. The fan had broken, and the device was presumably in danger of overheating. I would get a warning message each time I re-booted, and occasionally Windows would blow up. So shortly before Christmas, I bought an HP Envy Desktop, preparing to install it after my winter break. I did not buy a printer or monitor: I had an HP Photosmart printer that was working well, and, only a year ago, I had had to replace the monitor that had suddenly died on me with a new model. This new monitor had HDMI support, but, since my PC was so old, it did not support an HDMI connection, and I thus had to use the older-generation VGA connector. This apparently meant that I had no sound support on my computer, but that was no great loss, even though I could not listen to music while I was working. I got used to it. Early in January, I thus loaded up the printer with new ink cartridges, backed up the files on the old PC, checked the cable configurations to ensure I knew what socket went in where, and unpacked the new machine.

To start with, all went very smoothly. True, Windows10 was a bit of a shock, with some features apparently dropped, and some weird patterns of activity occurring, such as random duplication of keyboard strokes. But overall it worked, and I restored my files (well, partially: but that’s another story.) Then I suddenly realised that I was not getting any sound from the computer, despite the new HDMI connection. The driver was okay, the system told me that the graphics was working properly, and yet no sound emanated from the monitor. It took me a while to work out that, all that long year ago, I had been sold a monitor with no sound support. Well, it was my fault for not asking, I suppose, but I think the salesperson was at fault, as well. Maybe he just wanted to move that product off the shelf. After all, why would I want to move from an antiquated broken monitor that supported sound to a spiffy new one that didn’t? There’s a lesson.

Next, I tried the printer. And here is where the problems started. Word documents would not print at all; PDFs would print, but very faintly. Crossword grids from the Web printed out partially. Emails from my queue printed fine, however. (I have one to prove it, as it relates to my problems.) What was going on with my device, which had been working so well a day beforehand? What caused such erratic behavior, where some items came out fine, but others were ignored? I did not believe it was a dirty printhead problem (something I had encountered and fixed a couple of years ago). My first step was, on my next trip into Wilmington (thirty-five miles away) to go to Best Buy, the store where I had bought both the printer and the PC, and ask for their advice. They immediately said ‘buy a new printer’, hinting that many users suffered from the same or similar experiences, as it would be too expensive to investigate the problem, and printers were so cheap. But I wasn’t going to give up that quickly.

After looking on the Web for users with similar complaints, I tried a number of things. I reloaded the printer driver (the current version was dated October 2015, which was perhaps not encouraging). I deleted the device, and added it back in. I reset it. I set it up as a default printer. I tried printing test pages. At some stage I logged on to the Microsoft and HP support forums, where ‘experts’ (but not employees of the respective companies) would generously offer suggestions to fix the problem. Nothing worked. Eventually, an HP employee joined the forum, and tried to help me. I shan’t go through all the steps he recommended, but he ended up giving me secret codes to enter on the printer itself, to determine why it wasn’t able to operate any off-line functions either. But even this process did not work as he outlined, as it was interrupted by another message. At this stage, we agreed that I should call up HP customer support.

Since the problem appeared to be with my newly warranted PC, I called the number for desktop computers, and was soon speaking to a support representative (in India), to whom I gave all the relevant information. Then, when I described my problem, he said that I needed to speak to the Ink-jet support group, and gave me another number to call. I went through the same process, was given a case-number, and started providing details of my problem. But when I gave the representative the Serial ID of my printer, she (in the US, this time) told me that I would have to pay for support, as the device was no longer under warranty. This did not completely surprise me – I have paid for such telephone support from HP beforehand – but I was not actually in the mood, given the trials I had already experienced, for having to pay for diagnosis that I really felt was HP’s responsibility. I somehow convinced her that she should at least provide an initial investigation of the problem for free. So we downloaded some software that allowed her to control my computer while I watched.

What happened next was rather disturbing. The representative asked me what make of router I was using, and when I responded ‘Ubee’, she expressed a degree of shock, almost one of recognition, as if the Ubee-Photosmart combination was a known toxic one. I tried to determine whether that was the case, but received no reply, as she started manipulating the Ubee tables on my PC. Clearly, she knows what she is doing, I said to myself. And then the connections were lost. First, the phone contact disappeared. She sent me a message indicating such, so I quickly sent her a text, imploring her to call me back. Then that connection went dead, too, and I was left stranded, with the shape of my router tables unknown, and the problem unresolved.

At least I had a case number. I called back, but this time was routed to another call-centre in India. Even though I gave the representative there the case-number, and told him what had happened, he claimed he could do nothing for me. I rung off in exasperation, hoping that the contact in the USA would call me back. But nothing happened. I suspect that the supervisor of the representative trying to help me in the USA had interrupted the process, probably reprimanding the young lady for not charging me for such support time, and thus had broken off all contact. I shall never know. Even when an HP customer relations person (who had presumably kept an eye on the forum, and had been alerted by the HP technician who joined it) contacted me afterwards, he was powerless to find out what had been going on. But to abandon a customer half-way through a process when the device was under the control of a remote technician was scandalous, in my unhumble opinion.

So I gave up, and bought a new printer, from Epson. Never again any HP products for me.

Perhaps it was all a strange coincidence, but one afterthought came to me. If my printer had enough intelligence in it that, when I ran out of ink, and inserted new cartridges, it could send a message in real-time to HP Central to encourage me to buy a replacement set, maybe it was also smart enough to detect that it was now being driven by a more modern, faster computer, and that a process akin to what we systems engineers used to call ‘graceful degradation’ should occur, so that the user would have to buy a new printer? That was the immediate recommendation of the technician at the company who sold me the printer, remember. After all, Apple has admitted slowing down its devices to preserve battery power, and Volkswagen fudged emissions when engines detected that they were running under laboratory tests. I would not be at all surprised if something like that happened.

And then my wife’s laptop computer started having problems. She would be told that an important Security update needed to be installed on Windows10, after which the process would hog her computer for hours on end, only to fail with the message ‘0x800700c1‘, when it was 99% complete. We ignored it for a while, since I was mightily consumed with sorting out my own PC, but I at last got round to investigating. ‘Contact Microsoft Support’ was the guidance, so I went on-line, and was soon directed to a document titled “You receive the error message ‘Something went wrong’, when attempting to install the latest version of Windows10.” I was amazed to learn that the company offered ‘many steps that I could try’, as there were ‘many possible reasons your device may be unable to update to the latest version of Windows’. This was extraordinary. A specific error message had been issued, yet the software had no clue as to what circumstances had cause it to fail, and the user of a consumer product was supposed to experiment with all these approaches in order to resolve the problem? What on earth would the Little Old Lady from Dubuque do?

I decided to request an on-line chat with a support person. This did not take long, and I was put in touch with Parthiban, in India. We set up the protocol by which such persons take over control of the computer, and he soon decided that the problem was due to a corrupt database, and a conflict with Norton Security. He initiated the update again, but he had to sign off before the process completed, leaving me with a link that I could invoke in case of failure. I was given a case-number, and waited for an hour or so. And then the installation failed again. So the next day, I used the reinvocation, and was before long involved in another on-line chat, with Deepthi. Now Deepthi did not appear to know what he (or she) was doing, as I could watch him wandering aimlessly around HP configuration options. My mistrust was justified, as he suddenly signed off the session without letting me know why.

Accordingly, the next day, I reinvoked the link, and noticed that I was 93rd in line, so decided to try again later. The queue had then diminished to 21, so I tried it again, and was soon engaged in an on-line exchange with Praveen. His diagnosis was that some cookies needed to be removed, and Norton Security had to be disabled for a while, as it was inhibiting the execution of the Microsoft Update routines. So I watched as he cheerfully went through the whole process leading up to the installation of the updates. Then he left me to watch for an hour, until the update failed again.

Yet, when I tried to re-invoke the link to resume my interchange, I was told that it was no longer valid. This time, I resolved to speak to a real person, called the support number, and, after a wait of about fifteen minutes, I described my problem to the support representative. She took my number, and soon I was talking to another agent, named Tony. (By asking him what time it was where he was working, I determined that he must be located somewhere in the Mid-West.) Anyway, while he seemed to be unable to look up my Call Number, and discover what approaches had already been applied, Tony sounded much more confident, and judged that I needed a larger partition size to run the routines. So I watched as he downloaded the Minitool Partition Wizard (how come Microsoft does not supply this facility?), which ran for about half an hour. That task having been successfully completed, he said he was going to re-install the whole of Windows10, so that I would not have to deal with a separate Security Update. I was getting a bit anxious as this process started, so I begged him to stay on-line until it completed, indicating that he could multitask with other customers while the update continued. Yet he was so confident that his solution would work, he said we should ring off: he did however commit to calling me in another hour to check how things were going.

Predictably, the update failed. After about an hour and a half of installation, verification, preparation and execution, I received a short message, with no diagnostic code: ‘Windows installation has failed’. And this saga would not be complete unless I informed you that, no, Agent 4 (Tony) never called me back, despite his promise. I had been abandoned again.

Before finally agreeing to give up completely, and simply to ignore the messages emanating from Microsoft that were constantly bugging my wife, as she worked at her computer, informing her that her security was at risk, and that updates still needed to be installed, I decide to post a plaintive appeal on the Microsoft Support Forum. I summarized all that had occurred, and expressed my frustration at Microsoft’s shoddy installation software, and its even more unprofessional support agents, who appeared to apply guesswork in trying to resolve problems, and repeatedly left consumers like me hanging dry. My appeal was quickly picked up by a Microsoft employee who has been very patient in going through my experiences. Yet his final recommendation, after I gave him the status of my Windows10 System Build, and maintenance applied, sounded very much like the process that Agent 4 had undertaken. When I pointed this out, he urged me to try what was (he said) a very simple process: indeed, he himself had written the on-line document that guided it. So I sat down, went through his steps, disabling Norton Security and trying again when that package told me that one of Microsoft’s modules was unsafe, and had had to be removed. About ninety minutes later, the Microsoft software, having gone through download, installation, verification, and preparation, started its execution. After half an hour, I received exactly the same message that had appeared in the previous try: ‘Windows installation has failed’.

The Forum Observer responded promptly, requesting that I send him (via OneDrive) a couple of log files from an obscure Windows folder. I am not sure why no one had thought of inspecting such data before (I had in fact suggested such a course of action several days earlier, as I suspected such files should exist somewhere). I had not used OneDrive (Microsoft’s file-sharing service on the Cloud) before, but I retrieved the logs, followed the instructions from my iPad, created the OneDrive link, and posted it on the Forum page.

And then I received the following amazing message from the moderator:

“A Windows upgrade requires DISM utility to work and in your case DISM fails which then triggers a rollback.

Error initializing DISM Session: [0x800700c1], [gle=0 x800700c1]

Right-click Start>Command Prompt (admin) and type in:

DISM /ONLINE/ CLEANUP-IMAGE/ SCANHEALTH

If that fails with 193 post back the DISM log present at C:\Windows\Logs\DISM\ again through Onedrive”

As John McEnroe would say: ‘You cannot be serious!” And don’t you just hate it when your DISM fails? So I went ahead, and yes, the SCANHEALTH failed with a 193, and I posted on the forum the link to the DISM log on OneDrive. Isn’t this exciting?

The next news was not good. My contact thought that the damage ‘was beyond repair, and that I would either have to reset Windows or do a clean install. He pointed me to another link, where a Mr Carmack had published a document titled ‘Clean Install Windows 10”. Mr Carmack attempted to sell the process by describing it as ‘a game-changing learning experience that will make you permanently the master of my PC’, going on to write that ‘to stretch this out over days or weeks you’ll learn better how each change affects performance.’ But typical home users of PCs do not have ambitions of becoming geeks, taking up Windows maintenance as a hobby. The only game I wanted changed was the one of getting Microsoft to fix its software. The steps that Mr Carmack outlined are monstrous (see https://answers.microsoft.com/en-us/windows/forum/windows_10-windows_install/clean-install-windows-10/1c426bdf-79b1-4d42-be93-17378d93e587), and must be very prone to error. And, even if I went through all this, what were the chances the problem would recur? I replied in this vein, thanking the moderator again, suggesting it was perhaps time to give up. ‘So what are the implications of simply ignoring the attempts by Windows to install the Security updates? Maybe the laptop should simply be replaced?’, I asked.

There is an easier way, replied the moderator. He outlined some other steps, recommending that I do a reset, ‘as it might remove the corrupt driver which is preventing the upgrade’.  He had no idea what might have caused the problem, and suggested yet another site ‘where the experts might be able to help you better’. But, if a driver has been identified as defective, I wondered, why could it not be replaced? At this stage, I concluded that I had had enough. My wife and I would live with whatever nonsense Microsoft imposed on us, and replace the laptop with something from Apple when the time came.

It was difficult for me to imagine that my wife’s PC was the only one on the planet undergoing such experiences. She is a woman in a million, I know, but I do not understand how her rarity should extend to the tribulations on her laptop computer. And the exercise also reminded me how little way the software industry (or Microsoft, at any rate) has come in fifty years. The company delivers an upgrade to a system that is in many ways incompatible with the previous versions, and it has disabled certain functions. The on-line documentation frequently does not match how the screens of system information appear, so one is left groping. The diagnostic codes given when the software encounters problems are meaningless and obscure. One can find jokey tutorials on YouTube, but they are badly designed, often delivered in mumbles, and do not explain enough about the Whys of a particular feature. The support personnel who try to help the bewildered consumer are poorly trained, not provided with proper tools, and thus engage in guesswork. And, of course, we fogies have to deal with tracking down those tiny labels with product serial numbers, pasted in the most inaccessible places on the equipment, that have to be read with a magnifying-glass.

What galls me even more is that we (in the USA, anyway) are currently facing a bombardment of in-your-face advertising from Microsoft that promotes its new expertise in Artificial Intelligence as ‘Empowering Imagination’. It depresses me to think how such technology will be abused by a company so obviously inept at managing the release and maintenance of its own software. Perhaps the techniques of neural networks should be applied to Microsoft’s own configuration and diagnostic problems before they are imposed upon an unsuspecting world? Yet again, we have been here beforehand. I recall the surge of enthusiasm about AI about thirty years ago, when all number of hyperbolic claims were made about the advent of rule-based systems. Now we hear it again, with all sort of nonsense about systems that will be able to teach themselves how to be more effective, and thus achieve all manner of breakthroughs in medical diagnosis, or fraud detection, or whatever. Computers can be programmed to give results that appear to reflect intelligence, such as beating grandmasters at chess, but that does not mean that they are inherently intelligent.

Maybe this generation of AI is different, but a caveat remains. A key principle of computing science has been the verifiability of systems – the fact that code must be inspected to determine whether the logic has been implemented according to specifications. (If proper specifications actually exist, of course, which is a whole other problem: see Multiple Record Hold.) Thus I used to experience the process of ‘structured walk-throughs’, where one’s peers would wade laboriously through the code a colleague had written to apply more stringent tests that might escape the test data environment. If the onus of decision-making has now been delegated to the computing system itself, who now takes responsibility when something goes wrong? I was both amused and perturbed to read, in the New York Times, earlier this month, how engineers at Google have started analyzing how computers using neural networks reach the conclusions they do, as if the experts are concerned about the level of auditability that these systems provide. “Understanding how these systems work will become more important as they make decisions, like who gets a job and how a self-driving car responds to emergencies”, the article declared. (I write this the day after the Uber self-driving car in Tempe killed a pedestrian during a test-run.) Their concerns are appropriate: I smell litigation over unexplained, and inexplicable, disasters. The paradox is that, if the processes of AI are verifiable, the technology is considered mundane and unimaginative, while, if they are not, it is uncontrollable and dangerous.  What do you think, HAL?

*          *          *          *          *          *          *          *          *          *          *          *

A few years ago, the Times of London informed me it could no longer issue a cheque for the occasional fees for published Listener crossword puzzles without my submitting a complex form that confirmed that I was a proper US-resident tax-payer. The cost to complete the forms required was almost as much as the crossword fee, so I didn’t bother. Last year, my bank in the UK (with whom I have had an account since 1965) told me that I would have to change my deposit account into a long-term instrument that would mature in three years, as it was no longer allowed to pay interest on accounts to overseas customers. This month, I received a letter from Barclaycard (with whom I have had a sterling credit card for about forty years) advising me that my account would have to be closed in early April unless I could provide proof of a residential address in the United Kingdom. Thus another convenience (for paying magazine subscriptions, downloading files from the National Archives, purchasing gifts, even ordering a copy of my own book from amazon to send to a reviewer – all in sterling) disappears. I have maintained my UK citizenship, have paid all tax at source, as appropriate, and have always declared all my (puny) UK-based income to the US Internal Revenue Service. It is comforting to know that the British authorities are cracking down on the real risks to currency and tax fraud, and thus discouraging me from any further investments or expenditure in the UK, while allowing all that other soiled money from Russia and other places to be brought into London for the purposes of acquiring valuable assets and helping the economy.

This month’s Commonplace entries can be found here.

1 Comment

Filed under Personal, Technology

Web Woes

Last January, I received an email purporting to come from my bank. It looked legitimate: it had a clean logo, in the right colours, but it contained a predictable spelling mistake, and did not originate from a valid bank email address. Yet I was concerned how the sender had obtained the fact that I was a customer of the bank, and gained possession of my email address. Had there been a serious security breach? Having occasionally received spoof emails from other institutions, which I forwarded to the address they gave for reporting such, and subsequently received grateful acknowledgments, I did the same with this one. I looked up the address to which such suspected spoofs should be sent (abuse@  . .  .) , and waited for a response.

And waited. And waited. I lingered a couple of days, and then sent another message to that address, inquiring whether the mailbox was being monitored, and requesting a reply. There was still no response, or even an acknowledgment. That was depressing, and utterly unsatisfactory. I thus went to the website again, trying to find a manager responsible for email fraud. The website was singularly unhelpful: it did not allow any chatroom discussion of security topics, and I entered a hopeless loop of going back to being invited to send further emails to the given ‘abuse’ email address. The bank provided no lists of executives to contact, no bank head office address to write to, only a couple of telephone numbers, neither of which looked suitable for my problem.

I tried one of the numbers, and after going through security checks, I spoke to someone (in Ohio or Iowa, I believe). She could not help me, but agreed to forward me to someone who could. I was thus transferred to a number in Atlanta, where I again introduced myself and my problem, and went through security checks. That person also decided that he was not in the office that could help me, but knew which section was responsible, and transferred me to another number.

I waited about twenty minutes before someone accepted my call. I again described my problem, and went through the same security checks. I was then told that that office was responsible for ATM security, but not for possible spoofing breaches. When I described my frustration to her, she said that she did not know what the policy was, but it was maybe unrealistic of me to expect any response from the Abuse department. I replied that these days it was very easy to set up an automated email reply system that would at least confirm that a customer’s message had been received, and indicate what kind of action was being taken, and added that it seemed to me that the Bank did not look as if it took reports of spoofing attacks, and possible security breaches, very seriously. She assured me that that was not so, and agreed to track down the Abuse Department. I was then left hanging on the telephone for another five minutes.

When she returned, she gave me the name and address of a ‘Resolutions Services and Support’ office, but no telephone number, no name of an executive responsible, and could not explain why that was not so. When I asked her what I should do next if I sent a letter to that office, and received no reply, she encouraged me to write ‘Response Required’, to ensure that I did receive a reply. This I did. But I was not hopeful.

Fifteen years ago, when the Web started to become a useful communications mechanism, corporate websites were full of data about organisation, functions, executives, addresses, telephone numbers, etc. Nowadays, it seems that their prime purpose is to provide a blatant marketing presence, and to make it extremely difficult for the inquiring customer (or prospective customer) to identify a department or person he or she might wish to contact. In addition, we have the blitz of customised advertisements: I cannot bring up the BBC website to check the cricket scores, or surf to a news site to ascertain Kim Kardashian’s views on this year’s Man Booker Prize nominations, without waiting for half a minute while dopey high-resolution advertisements for car dealerships half an hour away, that I am never going to visit, are loaded. Somebody, somewhere, is paying for all this, and will one day work out that it is all a waste.

After composing a letter, and sending it to the address given, I had one last try at finding a real person’s telephone number. Eventually I found one, in the Public Relations department. I called it, and left a message describing my problem (it was a Saturday), thinking I had done all I could. And then, out of the blue, a couple of hours later, I received a very polite telephone call from a Bank employee, who said that he was the Executive in charge of Security. His friend in the PR department had picked up my message, and alerted him to it.

As we discussed my problem, Mr. Watkins (not his real name) apologised, but said that, owing to the vast amount of spear-phishing emails that the Bank received these days, it had decided not to acknowledge any messages received from its customers, as it only encouraged more traffic that could overwhelm the system, and he started to brief me on the security challenges that any bank of its size has to counter in 2017. I responded that that might be so, but in that case why did the Bank simply not include some text to indicate that it inspected every genuine message that came through to its hotline, but that it would probably not respond individually to every item? Would that not provide for a better management of customer expectations?

At this stage, Mr. Watkins started to give me another little lesson about technology, at which point I decided to explain my credentials. While I am no longer au fait with all the issues to do with website maintenance and data security, I was one of the two executives who launched the Gartner Group’s Security product back in 1999. When I described my background, Mr. Watkins became even more amenable, and we moved on to a new plane. He seemed very proud of the fact that the Bank spends millions and millions of dollars each year on security. He essentially agreed with my recommendations, gave me his telephone number, and encouraged me to stay in touch while he investigated the problem.

Over the next few weeks, Mr Watkins was jauntily positive. There had been meetings, attended by database administrators, web designers, lawyers, security experts, public relations people – even manicurists, for all I know. It was important that everyone had buy-in to this significant portal of the bank’s business, and every detail had to be examined. And then, early in March, he proudly told me that the new functions had been implemented.

But they hadn’t. There are two entries to the bank system – a public one, and a subsequent secure sign-on that leads to a private area where customers can maintain their accounts. The Bank had attempted to fix the public ‘help’ area, where they had incorporated the text I suggested (although they made an egregious spelling mistake in doing so, spelling ‘fraudulently’ as ‘frauduleny’), but they had not touched the private zone. When I pointed this out to Mr Watkins, he was incredulous, and eventually I had to send him screenshots to prove that those spaces existed. I gently pointed out to him that it was as if the Bank’s executives had never tried to log on to their system as retail customers. He was suitably chastened, and promised to get back to me. More meetings with lawyers and psychotherapists, no doubt.

Nothing happened for a while. I continued to perform my on-line banking, and regularly checked the ‘Help’ section of the secure banking site to see whether it had been fixed. On March 20, Mr Watkins wrote to me as follows: “I’m writing as a brief status update to let you know that the changes you’ve identified below are scheduled to be implemented within the next 2 – 3 weeks.  In addition, I’ve had our team perform a comprehensive review of all of our web pages to ensure as much consistency as possible.  I will update you again once the necessary changes are complete.”

I waited again. No update from Mr Watkins, so six weeks later, on May 2, I emailed him again, pointing out that the unqualified advice still sat there, unimproved, in the private area, but did confirm that the rubric in what was called the Security Center was now clean and (reasonably) correct. (It had new spelling problems: ‘out’ for ‘our’, but no matter  . . .) I gave him the url of the offending area. Because of some personal issues, he had to hand my message over to his personal assistant to work on. He was under the impression he had already informed me about the changes the Bank had made.

I had to start again with Christine (not her real name). After she sent me an email informing me that the changes had been made, and how I should report suspicious emails, I had to explain to her that there was a discrepancy between the two zones, and I informed her of the fresh spelling problem. “Thank you for the feedback,” she replied. “We are currently working with our teams to review and will keep you posted.” More teams, more confusion. Less chance of a correct fix. I remembered Charles Wang of Computer Associates, who said once that, when a programming project started to drag, he would take a person off the team, so that it would run faster.

Another few weeks passed by. On May 25, I emailed Christine, and copied in Mr. Watkins, asking where things stood, only to receive the following reply from Mr Watkins. “I’ve tasked the multiple teams involved in producing and delivering these web pages to pull together a broad effort to reconcile all content.  These teams are currently researching what this will involve and we plan to meet back with them to discuss their assessments during the week of June 12. Please rest assured that there are no idle hands involved in this work but given the significant size and complexity of this effort, I’m focused on a) updating any current pages while b) ensuring the proper controls are in place to ensure ongoing alignment and consistency.”

Well, ‘resting’ I probably was, but ‘assured’ did not exactly describe my composure. I waited again. And then, on June 21, I learned from Christine that a new executive had been brought in to ‘address the issue going forward’ (as opposed to ‘going backward’, I suppose). I was invited to join a conference call, so that my concerns could be addressed. I declined, however. I did not need a conference call, and I instead carefully pointed out again that, while the problem had been fixed in the Privacy and Security Center, the text had not been incorporated in the private area, for which I provided the link again. All that Christine did was to provide me with instructions on how I should use the Bank’s web-page to report problems (as if it were not supposed to be self-explanatory by now).

I took one final stab at explaining the problem, pointing out how badly designed the whole website was, with its circular paths and inconsistent terminology, and I provided an explicit analysis of the problems with the Bank’s customer interface. I expressed my amazement that Bank officers could not identify the anomalies in the system, and fix them. I copied the message to Mr. Watkins.

On July 1, a new communicant appeared – probably not the executive brought in by Mr Watkins, as he introduced himself as being ‘on the team that oversees the on-line banking platform’. Arthur (again, not his real name) kindly provided me with a long explanation of all the changes that the Bank was introducing, including not just my recommendations, but many other improvements, as well. I thanked him, and promised to keep my eye open.

Well, it is now July 25, as I write, and the same old text appears under ‘Report Fraud’ in the private banking section, with no indication that messages will not be acknowledged. A simple change that I could have implemented on my own website in under five minutes (literally) still baffles the combined expertise of the Bank after seven months. Is this a record? Banks complain that they are stifled by regulation, but if they cannot even manage changes of this magnitude off their own bat, what hope is there for them? Is this story not an example of corporate incompetence and internal bureaucracy gone mad?

*                   *                  *                      *                      *                      *

The second incident concerns a recruitment at my old Oxford college, Christ Church (an institution, I hasten to add, for the benefit of my American readers, that is not actually the equivalent of Oral Roberts University, despite its name). The Hilary Term issue of the college magazine proudly announced that Christ Church was welcoming Sir Tim Berners-Lee as a Research Student and member of the Governing Body, with a mission to ‘grow Computer Science at Christ Church’. For those readers who might not know about Sir Tim’s remarkable achievements, I point you to https://en.wikipedia.org/wiki/Tim_Berners-Lee. He is known as the ‘inventor’ of the World Wide Web, and director of the World Wide Web Consortium, and took his degree at Queen’s College, Oxford. As a retired information technologist, I admire and applaud his achievements.

Yet some things that Sir Tim wrote in this promotional piece in Christ Church Matters puzzled and disturbed me. He characterised ‘several connected initiatives’ in which he has been involved for some time as Open Data, Open Standards, and Human Rights on Web. As an expert in data management for some decades (I was a data and database administrator in the 1970s, have experienced several generations of data-base management systems, was the lead analyst and product director for Strategic Data Management at the Gartner Group for a decade, and successfully forecast how the market would evolve), I believe I understand fairly well the issues regarding data security and data sharing. I found Sir Tim’s pronouncements about Open Data naïve and erroneous, and his thoughts on the role of Open Standards confusing, and maybe misplaced. But what really provoked me was what he wrote about Human Rights on the Web. “We have a duty to ensure that the Web serves humanity, and all of humanity”, he wrote, adding, somewhat rhetorically, about the concerns of the Foundation: “Is it [the Web] open, non-discriminatory, private and available to all, including minorities and women? Is it a propagating medium for truth and understanding, or more so for untruth and discord? Can these parameters be changed?”

Now I regard such questions as reasonably interesting, although I’m not sure what ‘minorities’ he was referring to (philatelists? Zoroastrians?), or why ‘women’ should come at the end of his list of concerns. But how could computer science be sensitive to such transitory social labels, or the gender of its users? Quite simply, what he proposes is either outside the realm of computer science, or lacking any toehold in what computer science has already generated about issues of data management (for instance, in the works of Sir Tim’s outstanding forbear, Edgar Codd, another Oxford man, an alumnus of Exeter College, and also a winner of the Turing Award). I found his pronouncements about serving humanity simply arrogant and pompous. Accordingly, early last March, I wrote a letter to the editor of Christ Church Matters, and to the Dean (whom I met last year, as my blog reported), which ran as follows:

“Am I the only reader of Christ Church Matters to be somewhat surprised, even alarmed, at the expressed rationale behind the new computer science initiative? The achievements of Sir Tim Berners-Lee are spectacular, and I have no doubt his intentions are honourable, but do the goals that he espouses not tread on the space of social advocacy, even corporate mission, rather than scientific investigation?

For example, the notions of ‘web-based data’, ‘Open Data’ and that ‘we [= who?] have a duty to ensure that the Web serves humanity, and all of humanity’ are certainly controversial. Data are not exclusively managed by web applications, but frequently shared. Indeed, it is a principle of good database design (a topic frequently overlooked in university computer science courses) that data be implemented for potential shared use, irrespective of delivery vehicle. There is thus no such entity as ‘Web-based data’. Professor Wooldridge’s statement that ‘when Governments generate data, there is huge potential value of that data is made freely available and open for all to use’ provokes enormous questions of privacy and security. To assume (as does Sir Tim) that ‘we’ can be confident enough to know how ‘all of humanity can be served’ has a dangerously utopian ring to it. Etc., etc.

The point is that technology is neutral: it can be used for good, or for ill, effect, and people will even disagree what those two outcomes mean. How is ‘all of humanity’ served when Islamic fanaticists can exploit web-based encrypted information-sharing applications to exchange plans for terror? Who benefits when private medical data is presumably made available for ‘all to use’? When is data private and when open? It is all very well for Sir Tim to assert that that his main motivation is ‘the personal empowerment of people and groups’ (is that phrase not both tautological and self-contradictory?), but that is a belief derived from his own sense of mission, not from a perspective of scientific inquiry.

Maybe these matters have already been discussed, and have been resolved. If so, I think it would be desirable to have them explained publicly. I believe those helping to fund such initiatives should be made aware that the boundary between science and evangelism appears to have shifted considerably.”

My letter was kindly acknowledged by the Dean, with a promise of follow-up, but I have heard nothing more. I suspect that I am seen as a minor irritant, getting in the way of some serious boosting of the college reputation, or maybe hindering access to vital government funding. But the question remains. There are researchers into computer science, and there are commercial enterprises. They frequently enjoy a symbiotic relationship, but there comes a time when enterprise have to make risks and decisions that go beyond what consortia and standards-groups can achieve. Ironically, Sir Tim’s statements about benefitting humanity sound uncannily like those of Mark Zuckerberg, the CEO of Facebook, who also has evangelical designs on improving the world. But the rest of us should be very wary of anybody who claims that omniscience to know how ‘humanity’ is best served, and who appears to be unaware of the Law of Unintended Consequences. And computer scientists should not start dabbling in evangelism.

*                      *                      *                      *                      *                      *

Regular readers of this website will recall my reference to The Trinity Six, by Charles Cumming, in my March blog. Since then, I have read his first Thomas Kell novel, A Foreign Country, and this month, the follow-up A Colder War (published in 2014), both of which I recommend. (Although I do not understand why we need to know every time Thomas Kell lights up a cigarette, or that he throws the butt of one into the Bosporus.) But my point here is to describe how unmistakably set in time these thrillers are – not so much by the political climate, although Iranian nuclear secrets and rebellious Turkish journalists give one a sense of that  ̶  but more by the use of technology. For the narrative is densely imbued with BlackBerries, iPhones, Facebook, TripAdvisor, SIM cards, SMS and O2 services  ̶  but not the dark Web, Snapchat or Twitter (or even Sir Tim’s Open Data initiative). Will it make the book soon seem dreadfully outdated, or will it be praised for its verisimilitude?

The pivot of the plot is indeed one such technological matter. (Spoiler Alert.) In what appeared to me as a very obvious mistake by the hero, an unencrypted text message leads to the eventual betrayal. And one other passage caught my eye  ̶  for a different reason. Cumming writes, about a surveillance operation at Harrod’s: “While most of the members of the team were using earpieces and concealed microphones, Amos had been given an antediluvian Nokia of the sort favored by grandparents and lonely widowers. Kell had banked on the phone giving plausible cover.”

I recognized that scene. Three or four years ago, I went into a branch of my bank to pay in a cheque (it may have been a check). The cheerful spirit behind the counter asked me whether I knew that I could pay in checks via my cell-phone (or mobile, as it would be known in the UK). Without saying a word, I then solemnly produced my venerated Motorola C155, manufactured ca. 2005, reliable, rugged, and not very handsome, and showed it to the woman. She then let out an enormous giggle, as if to draw the attention of her co-workers to this antediluvian instrument. As can be seen, it looks more like the shoebox phone from Get Smart (the 1960 TV series, not the 2008 movie).

But it did its job – just made and received phonecalls. My carrier forced me to replace it a couple of years ago, but, my fingers are too stubby for the keypad on the new thin model, and I never use my phone to access the Web. Enough woes in that. I miss my C155  ̶  ‘as favored by grandparents’.

*                            *                      *                      *                      *

Another saga started. In May, I had received a letter from History Today, inviting me to renew my subscription on-line. “Renewing your subscription couldn’t be easier”, it boasted.   I thus logged on to its website, but was frustrated in my attempts. I sent an email to the publisher, listing my failures. I explained that the system did not recognise that I was in the USA, did not allow me to enter my subscription reference, and quoted a sterling fee rather than the $99 mentioned in the letter. And when I signed on to my account, it gave me no option to renew, just to upgrade to access to the archive.  I received a prompt reply, which merely stated that the website had been going through some maintenance, but that once this were completed, I should be able to renew my subscription on-line.

I held off for a while, and then received another letter in the mail, which again proclaimed that ‘renewing your subscription couldn’t be easier’. It offered a price of $79, which I interpreted as a special offer, maybe making amends for the earlier technical problems. I thus logged on afresh, and made the renewal, but did notice that the confirmation came through with a charge against my US dollar credit card for £99. An obvious mistake, no doubt to be cleared up simply. I sent an email pointing out the error. After a couple of days, I had received no response apart from an email confirming my renewal, and encouraging me to contact the sender (the third name in as many messages) if I had any problems. I thus sent off another email, pointing out the discrepancy between the amount specified in the invitation letter, and somewhat impatiently requested a credit to be made against my credit card.

Yet another name replied, with the following message: “Thank you for your recent email.
I can confirm the reason they are different amounts and different currency is because it has been converted from USD to Pounds. So it will always show what we have received as payment here is England rather than the amount you paid is Dollars. If there is anything else that I can help you with please don’t hesitate to contact me.”

So, as the month wound down, I sent another message, pointing out that a fee of $79 would convert to £61, not £99. I am awaiting their reply. It is possible, I suppose, that they mistakenly took the exchange rate as 1.31 pounds to the dollar, rather than vice versa, although the letter lists the optimal online archive upgrade as a more accurate £30/$45. We shall see. If e-business speeds are predictable, I shall probably be able to provide an update to this transaction in January 2018.

The next episode of Sonia’s Radio will appear at the end of August. This month’s new Commonplace entries appear here.

Leave a Comment

Filed under Literature/Academia, Media, Personal, Technology

On Privacy and Publicity

While reading Robert Tombs’ superlative The English and Their History, I came across the following sentence, describing Samuel Johnson’s and Richard Addison’s London: “The mix of commerce and culture produced what has been termed ‘the public sphere’ – places and institutions for exchanging information and forming opinion, which lay between the purely private world and the official realm”. What could be more representative of that sphere in twenty-first century Britain than the pages of Prospect magazine, ‘the leading magazine of ideas’, as it promotes itself?

The February issue of Prospect included an article that outlined what has to be done with technology – primarily that concerning the use of social networking – to keep the citizens of the UK safe while protecting their liberties. The following earnest and superficially innocuous paragraph caught my eye: “The big technology companies have a crucial role – and unique responsibility – in building the security that keeps us free and safe. We trust them in part because they are private. Co-operation is much preferable to legislation. The next step is for all parties to collaborate on a way forward to benefit from new technologies while doing what we can to stop those who would do us harm. This kind of co-operation between public and private sectors is needs in free societies where security underpins our privacy, private enterprise and liberal democracy.”

But this simply will not do. To begin with, this contrast of ‘the public sector’ and ‘the private sector’ is hopelessly naïve. Whereas a government (or its civil servants) may be said to represent the populace, there is no such entity as ‘the private sector’ that may be negotiated with. A free market consists of a number of competing entities trying to differentiate themselves. Politicians frequently display a very wooden understanding of how markets work: I recall David Cameron’s meetings with ‘industry leaders’ to discover what it is they need from government. But what today’s leading businesses want will be protection in some way from any upstarts who threaten their turf. The needs of the market are not the same as the needs of current market-leaders. (Think of Norwegian Airlines threatening the established transatlantic carriers.) The FBI made the same mistake in thinking it could negotiate with ‘Silicon Valley leaders’ as a method of resolving this problem of encrypted information on PDAs and cellphones. This echoed the policy of President Obama, who in 2015 made a point of trying to ‘cooperate’ personally with Silicon Valley on these issues. Just this week, Obama officials again met representatives from technology and entertainment companies (but not chief executives) to discuss ways of combating extremists on-line. They still do not get it. This is a matter of law – to be addressed either by an interpretation of existing laws, or by new legislation. Parliament, not parleys.

For example, had a similar advance been suggested to computer technology leaders twenty-five years ago, the list of vendors would have probably included IBM, ICL, Data General, DEC, Wang, Honeywell, Siemens-Nixdorf  . . .  Apart from IBM, where are they now? Apple is presumably the IBM of today, but there is no guarantee that the ‘big technology companies of today’  (e.g. Facebook, Google, Snapchat, Twitter and Buzzfeed? – my computer industry advisory panel supplied me with these names) will dominate in ten years’ time. How long ago were Nokia and Blackberry the leaders in personal networking, for example? So how can such a suggested initiative encompass the coming vendors of tomorrow? Schumpeterian creative destruction is always at work.

What’s more, it would be illegal. Since most of the companies affected are American, any move by such to meet to discuss shared endeavours would have to be considered under anti-trust legislation (something that should probably have taken affect with Obama’s Affordable Healthcare Act, by the way.) For such companies to ‘collaborate’ with government to define pseudo-voluntary technology ‘standards’ (that would then be implemented at the whim of each company’s R & D design and implementation schedule) would be called for exactly what it is – conspiracy. And this aspect does not even touch the issue of whether such measures would be effective – which I shall not get into. This issue has been gaining intense attention in the past month, when Apple’s Tim Cook has again been assailed by the US Department of Justice. Cook has spoken out vigorously with the opinion that any back-door capabilities into a supplier’s encryption system would be abused by the bad guys. At the same time, Apple is planning for greater encryption of customers’ data in its ‘cloud’, which will make things even more difficult for law enforcement. (‘Ou sont les nuages d’antan?’) Yet in an Op-Ed piece in the New York Times on February 23, William J. Bratton and John J. Miller gave as their concluding argument for demanding that Apple should unlock its iPhone that Google and Apple ‘handle more than 90 percent of mobile communications worldwide’, and thus should be accountable for more than just sales. If such a rule does apply, it should apply to everyone.

So who is the supposed expert making this fanciful suggestion of bonhomous co-operation? Step forward, Sir John Sawers, ex-head of MI6, who indeed wrote the article. Not only that, Sawers advertises himself as having been ‘Chief of the Secret Intelligence Service (MI6) between 2009 and 2014’, and his second paragraph reminds us immediately of his credentials: “As MI6 Chief, my top priority was identifying terror attacks against Britain planned from abroad.” Sawers is then described as being the Chairman of Macro Advisory Partners.

What in heaven’s name is the ex-head of MI6 doing exploiting his past career while claiming to be an independent consultant? And how can he suggest that his role therefore gives him some credibility in representing the requirements and desires of the ‘public’ sector? There cannot be a more private organisation than MI6, whose very existence was withheld from the British public until 1994, of which no archival material has been released after 1949 (the year where the authorised history stops), and whence any retiring head a decade or two ago would have quietly folded his tent, picked up his ‘K’ (although Sawers had that already), and shimmied off to Torquay to tend his geraniums and take up square-dancing. Now such persons write their memoirs – surely in contravention of the Official Secrets Act  ̶  and pontificate with the chattering classes in the press.

This dual role of subtly promoting MI6 connections and policy, and claiming to be an independent advisor, does not sit well with me. Can MI5 and MI6 not speak openly themselves about such policy? What do they think of this grandstanding and self-promotion, I wonder? Or has Sawers undergone some shift in position now that he has left his official intelligence hutch behind? If so, shouldn’t he describe what that is?

It gets worse, in a way. A quick search on the Web for Macro Advisory Partners shows that the firm has a Global Advisory Board of seven (see http://www.macroadvisorypartners.com/the-firm/global-advisory-board ), of whom the prominent names are Kofi Annan (seventh Secretary-General of the United Nations), David Milliband (of Labour Party renown, and now President and CEO, International Rescue Committee), and William J. Burns (President of the Carnegie Endowment for International Peace, an institute which regrettably sounds like one of those Soviet fronts of the late 1940s: indeed, the Soviet spy Alger Hiss was its President between 1946 and 1949.) I didn’t see Cherie Blair’s name there yet, but she is no doubt a very busy woman. Sawers was Britain’s permanent representative to the United Nations between 2007 and 2009, so he no doubt developed some good contacts then. But is he running the show, or he taking his advice from this group of Kumbaya do-gooders? How will his undoubted steeltrap mind have been affected by such company? No wonder his recommendation for solving the technology problem is to get everyone around a table in peace talks.

I believe this is all highly irregular. Sawers surely has a pension that he can live off comfortably: he does not need this jump into the ‘private’ sector, where, ironically he can be much more expansive about his ideas than he was when working for the government. The undoubted impression that casual readers will gain from this promotional journalism is that there is some consistency in MI6 policy from the Sawers regime to the current set-up. That must make it very difficult for the present leaders of MI6 – and MI5, of course – to develop policy and work it through the normal processes, dealing with this distracting noise in the media. If they agree with what Sawers says, are they admitting that they are likewise influenced by pollyannaish internationalist wishful thinkers, instead of by steely pragmatism? And if they disagree with him, what does that say about continuity of purpose and perspective within MI6? It is all very messy, and, in the jargon of today ‘unhelpful’. Sawers should not have been allowed to exploit his past experience for monetary gain, and should have been prevented from entering the public sphere in this way: his employers should have insisted on a more stringent termination agreement.

Lastly, all this reinforces the unhealthiness of the transfer of careers between government and industry, and also demonstrates how absurd the UK Honours System is. ‘Captains of industry’, managing directors of private companies publically traded, should be looking after the interests of their shareholders. They do not provide ‘services to the industry’, for which gongs are awarded.  In addition, they have their own generous rewards, being almost without exception overcompensated by crony boards of directors, and remunerated handsomely even if they fail. Public ‘servants’ (who all too often act as if they were our masters) should be expected to perform their jobs well: if they do not, they should be fired. And when they retire from highly-important positions, they should do exactly that – retire.

*          *          *          *          *          *          *          *          *          *          *          *

Readers who followed my representation to the New York Times in my December blog may be interested to know of the follow-up. Having gained no satisfaction from the Public Editor (Margaret Sullivan), I wrote an email to the Executive Editor, and then one to the CEO, Mark Thompson. These attempts having resulted in not even an acknowledgment, I then sent a letter to Mr. Thompson, with a copy to the publisher, Mr. Sulzberger. Again, I have failed to extract even an acknowledgment from either gentleman. Did Mr. Thompson learn such manners at Merton College, I wonder?

I have since challenged the Public Editor on the Times’s somewhat irregular decision to give Madeleine Albright the opportunity to explain away her Clinton election campaign gaffe (about women supporting other women lest they go to hell) in an Op-Ed column. Again, no reply. And then, Ms. Sullivan announced earlier this week that she was leaving the position early to join the Washington Post. Am I entitled to imagine that perhaps she became frustrated in dealing with the bizarre journalistic principles at the Times, and that the paper’s failure to act on my complaint pushed her over the edge? (‘Dream on, buster.’ Ed.) As for Mr. Thompson, he left a mess behind at the BBC, and I expect further messes at the Times. This week, the paper ran a story about the post-mortem at the BBC over the matter of protected ‘stars’ like Jimmy Savile, who were allowed to get away with sexual malpractices in a corporate culture of fear at a time when Mr. Thompson was Director-General of the BBC (2004 to 2012). Mr. Thompson’s responsibility for that culture – or even the fact that he led the organisation –  was omitted from the article.

In conclusion, I highlight an item from this month’s Commonplace entries, taken from Hugh Trevor-Roper’s waspish Wartime Journals: “The Christ Church manner, that assumption of effortless superiority, is said to be galling to those who weren’t at Christ Church. But we can’t expect the world to be run for the benefit of those who weren’t at Christ Church.” Indeed.  Stop looking shifty, Thompson.                                            (February 29, 2016)

1 Comment

Filed under Espionage/Intelligence, Politics, Technology

The Myth of Buying Market Share

A few years after I became an analyst/consultant at the Gartner Group, I was introduced by one of the DBMS vendors to the thoughts of Geoffrey Moore, who had some original ideas about the challenges of high-tech companies in introducing their disruptive products to mainstream buyers. His book, ‘Crossing the Chasm’ (1991) quickly became a classic in technology circles (see https://en.wikipedia.org/wiki/Crossing_the_Chasm), and I adopted his ideas in evaluating and guiding the strategies of companies in my bailiwick. Some CEOs claimed to be familiar with the theories, and even to putting them into practice, but since the distinct message in the early years of the Technology Adoption Life Cycle was ‘focus’, they understandably struggled to keep their companies in line. ‘Chasm’ thinking requires a proper marketing perspective, but independent VPs of Marketing in technological start-ups are a bit of a luxury, and VPs of Sales always think of Marketing as something that supports their Sales Plan, rather than of their Sales Plan as something which realizes the Marketing Plan. Trying to close a deal to unqualified and unsuitable prospects is frequently an exciting challenge for such types.

As my career at Gartner wound down, and I considered retirement, I chose to move to a small software company in Connecticut. I was quickly brought down to earth: as a Gartner consultant, I had earlier been engaged by the company for a day’s work, at quite high fees, during which the managers attending dutifully wrote down all I said, and nodded appreciatively. When I became an employee, however, and started suggesting (as VP of Strategic Planning) to the CEO how she might want to change some of the processes (such as not having the R & D plan changed each month after the latest visit by a customer or prospect to the development facility in Florida), I was swiftly told: ‘You don’t understand how we do things around here, Tony’. That was not a good sign. So I picked up my thinking about Chasm Crossing, tried to talk my CEO out of an acquisition strategy (devised to show muscle to the Wall Street analysts, but in fact disastrous), and reflected on how financial analysts misled investors about markets. I had learned a lot from the first software CEO I worked for, back in the early 1980s, but he was another who didn’t understand the growth challenge. ‘Entrepreneurial Critical Mass’ was the term he had used to persuade his owners to invest in an acquisition strategy that was equally misguided: I had had to pick up the pieces and try to make it work.  (This gentleman was also responsible for bringing to the world the expression ‘active and passive integrity in and of itself’ to describe the first release of a new feature, which presumably meant that it worked perfectly so long as you didn’t try to use it.)   My renewed deliberations now resulted in an article, titled ‘The Myth of Buying Market Share’, which explained how completely bogus estimates of ‘market size’ misled CEOs and investors into thinking that all they had to do to be successful was to pick up a portion of a fast-growing ‘market’. I believe it was published somewhere, but I cannot recall where.

I reproduce the article here. I have not changed a word: it could benefit from some tightening up in a few places, and some fresher examples, but otherwise I would not change a thing, even though it is now sixteen years old. At the time I wrote it, I contacted Geoffrey Moore, and sent him the piece. We spoke on the phone: he was very complimentary about my ideas, and we arranged to meet for dinner in San Francisco, where I was shortly to be attending a conference. I vaguely thought that I might spend my last few years actually putting into practice some of the notions that had been most useful to me in my analyst role, and wanted to ask Moore about opportunities at the Chasm Group. So, after the day’s sessions were over, I approached him, introduced myself, and said how much I was looking forward to dinner. He was brusque – dinner was off. Obviously something better, somebody more useful, had come along. I was for a few minutes crestfallen, but then realized that I would never want to work for someone who behaved that rudely. I resigned from the software company a month later and began my retirement a bit earlier than planned. Since then I have never touched the industry again, apart from one day’s work for another small software company in New Jersey that desperately needed help, and wanted to hire me as VP of Marketing after I did a day’s consulting for them. North Carolina beckoned, and I have never regretted getting out when I did.

After receiving a fascinating observation from a reader (via Nigel Rees), I have posted an update to my piece on ‘The Enchantment’. The normal set of Commonplace items can be found here.                                                                                                                   (January 31, 2016)

2 Comments

Filed under Economics/Business, Personal, Technology

The Congenial Richard Dawkins

When I was in my early twenties, I read a book titled something like ‘Why Darwin Is Wrong’. It wasn’t a creationist text, but a popular science-based analysis. I can’t find the volume on abebooks (which doesn’t appear to list anything before 1981), but I recall quite clearly two of its major objections to Darwinian thinking, so far as the author understood it. One, that the notion of ‘The Survival of the Fittest’ (which was actually coined by Herbert Spencer to describe Darwin’s natural selection) was tautological, and thus meaningless, since what was ‘survival’ but another way of saying that an animal was ’fit’?  Two, that if the energies that contributed to survival took place after the animal had passed on its genetic material to its offspring, there would be no mechanism by which more adaptive traits would endure in the species.

I thought at the time that these points had merit, yet I was not completely discouraged from accepting that natural selection was the most plausible explanation for evolution, even though the exact mechanisms by which it occurred were still somewhat mysterious. I was, however, dismayed by another misconception, namely the way that the Theory of Evolution was frequently misrepresented as something purposeful by even the most knowledgeable of experts. I can recall the great David Attenborough, in Life on Earth, explaining certain phenomena in terms such as: “Thus, in order to survive, the bats had to develop radar.” This notion of purpose in Evolution is obviously nonsensical, and I have occasionally had to write to the Science Editor of the New York Times to point out where their journalists mistakenly ascribe this sense of an objective to adaptive changes. After all, did certain winged birds develop their flightlessness in order to make their life less hazardous? And what was the timescale according to which such adaptive changes worked? How long would it take for various initiatives to fail or succeed before the lack of ‘fitness’ wiped out the species? At the same time, as Jonathan Weiner’s The Beak of the Finch showed, describing the researches of Peter and Rosemary Grant on the Galapagos, small changes in the dimensions of finches’ beaks could rapidly take place in the light of changing climatic conditions and food supply.

Then Richard Dawkins’s Selfish Gene came along and changed everything, showing that the gene, not the individual organism (as Darwin believed) was the unit of natural selection. I have enjoyed Dawkins’s books since, although I found his first volume of autobiography, An Appetite for Wonder, rather scrappy and chippy. Now I have just finished his sequel, Brief Candle in the Dark. This is a new Dawkins. I think his PR firm must advised him not to be so offensive and controversial, because he positively oozes congeniality, and is nice about nearly everybody, and not nearly as scathing about religion as he used to be. (There must be a social meme in such superstitions that aids the survival of certain groups, a sad but unavoidable truth.) He also turns out to have almost as many friends as did Denis Healey or Lord Weidenfeld, and appears at times unbearably smug. As a curmudgeon myself, maybe I preferred the traditional Dawkins.

He has some fascinating new insights about the evolutionary process. I was interested to see what he had to say about the hot topic of epigenetics (defined in Chambers as the ‘gradual production and organisation of parts’, which is the study of how gene behavior is affected by environmental factors), and how he contrasted it with preformationist thinking (i.e. that, in essence, a homunculus was inside every human embryo). It seemed to me lately that some neo-Lamarckians, interested in promoting the notion of the passing on of acquired characteristics, have latched on to the term of ‘epigenetics’ to assist their cause. A footnote (p 402) from Dawkins is worth citing in full: “Don’t by the way be confused by the fact that the word ‘epigenetics’ has recently been hijacked as a label for a fashionable and over-hyped idea that changes in gene expression (which of course happen all the time during the course of normal embryonic development, otherwise all cells of the body would be the same) can be passed on to future generations. Such transgenerational effects may occasionally happen and it’s a quite interesting, if rather rare, phenomenon. But it’s a shame that, in the popular press, the word ‘epigenetics’ is becoming misused as though cross-generational transmission was a part of the very definition of epigenetics, rather than a rare and interesting anomaly.” Thank you, Professor. Just what I was looking for.

In one area however, I wonder whether Dawkins has got it wrong. I recall, at about the same time that I read the book on Darwin, taking in another work that pointed out how quickly scientists make analogies between the human body and whatever the current state of technology is (i.e. a pump in the 17th c., a clock in the 18th , an engine in the 19th , a computer in the 20th ). I thought that it might have been Arthur Koestler in The Ghost in the Machine, but I can find no trace of it there, and in those pre-spreadsheet days I did not keep track of every book I read. No matter: I think the point is valid. And Dawkins falls into the same easy motion. On page 382, when discussing the possible source of language, he makes the claim that ‘the human brain must possess something equivalent to recursive subroutines’ (an ability for a computer program to call itself and then return to an outer version of itself), a feature he says exists in Algol 60, but not the original IBM Fortran  language he used. Such a feature in human genes, which he calls ‘macro-mutation’ might have come about in a single mutation, and could have been responsible for the ability to create the phenomenon of language syntax. In reducing a complex organic process to a mechanical one, however, I believe Dawkins makes a categorical mistake. A computer program is only an artifact of the entity that he is describing, namely the human brain, which is a far more complex phenomenon than the strings of ones and zeroes that comprise a language compiler. His comparison is therefore merely crude reductionism.

But then Dawkins compounds his error. He goes on to write: “Computer languages either allow recursion or they don’t. There’s no such thing as half-recursion. It’s an all or nothing software trick. And once that trick has been implemented, hierarchically embedded syntax immediately becomes possible and capable of generating indefinitely extended sentences.”  First of all, if it is a design feature, it is not a trick. The trick – if there were one – would be an inherent flaw in the software where recursion did not work properly all the time – either by faulty implementation, or by a deliberate clandestine approach that made aberrant decisions based on some external circumstance or internal control data. After all, we each one of us know now about the Volkswagen Emissions Control Software, which gave false readings when the engine was being tested under laboratory conditions. Similarly, the implementation of a compiler program that claimed to allow recession could disable the function, or cause it not to work properly, depending on, for the instance, the date or time of day, the machine environment, or the particular iteration or count of the software execution.

He thus fails to distinguish between the design statement for a compiler that allows recursion, and the instantiation of that design in code. Moreover, no software is a perfect implementation, which causes the analogy inevitably to stumble. And by hinting at the notion of design in computer languages (what he signifies as the ‘trick’), Dawkins inadvertently undermines his analogy, since that notion of an architect has no role to play in evolutionary development, natural selection being an essentially haphazard process. Too many of his metaphors (for example, the arms-race, p 340; or ‘if we think of natural selection as a sculptor’, p 359) contain this notion of design at work, and thus weaken his whole argument, since the congenial atheist would assuredly deny the role of any ‘Designer’ in the process of language evolution. While many of the mechanisms by which genetic change occurs are still mysterious, that does not mean they are mystical. Following up on this theme, Dawkins later goes on to praise Chomsky’s idea of the language-learning apparatus being genetically implanted in the brain – which also strikes me as a bogus concept, since so many languages have implementations of syntax that are utterly antithetical and incompatible with other schemes. This is the weakest part of Dawkins’s theorizing.

Still, it was all a stimulating and enjoyable read, if you can put up with Dawkins continually reminding you how clever and successful he has been.

P.S. The New York Times informed me, on November 25, that the Saeed Book Bank in Islamabad, Pakistan, sells a thousand copies of Dawkins’s atheist treatise ‘The God Delusion’ each year. Not many people know that.

New Commonplace entries appear here.                                                                                                     (November 30, 2015)

4 Comments

Filed under Personal, Science, Technology

‘All The News That’s Not Fit To Archive’

We relational database people are well-organized, methodical. We like analysis and business rules, strong notions of identity , the use of sets and non-significant keys, normalized designs and value-based links, precise versioning and time-stamps, and careful promotion of systems into production, with secure fall-back procedures. All that is tech-talk, but it means something in the real world. (One of the first articles I had published, back in 1980, in Datamation, was titled ‘The Importance of Good Relations’, which showed the link between solid database design and flexible business practices.)

Yet the Web has changed all this. When I first developed my website, under Microsoft’s FrontPage, there was some semblance of a test environment and a production environment. I would develop the site on my computer, and when I was ready, and had made sure all the links were defined, and pointed to real pages, I would upload the whole kit and caboodle to the host site, where the new system would replace the old, giving me the option of importing all pages that had changed (but admittedly with no easy fall-back to the previous version). No more. I now use something called WordPress, which I invoke on a remote server. It allows me to compose and save drafts of individual pages, but it is otherwise tightly integrated with the production system. If I promote a new page, it goes live immediately, and if I change it again ten seconds later, the page is immediately replaced, with the previous one lost for ever. (Unless it found its path to some entity called the Wayback Machine, which is described in a fascinating article by Jill Lepore in the New Yorker of January 26, 2015, titled The Cobweb: Can the Internet be archived?)

I mention all this in connection with my last plaint from the January blog, about the New York Times, and its practice of making changes to its electronic versions of articles after they have been published in the printed version (or the late printed version, since that happens, too. We in North Carolina get an earlier version than the people up in New York, for example.) The reason this concerns me is primarily one of research integrity, since there is no longer a ‘paper of record’ on which historians can rely. I made this point in an email to the Public Editor, whose office eventually acknowledged my inquiry, promised to look into it, but then withdrew in silence. So, after a couple of weeks, I checked out the paper’s Statement of Standards and Ethics, and wrote to the Vice-President of Corporate Communications. The essence of my message ran as follows:

“For there is a vital question to be answered: ‘What is the paper of record?’ Your slogan on the first page of the printed edition is still ‘All The News That’s Fit To Print’, but apparently some of that news is Not Fit To Archive. What happens when historians attempt to use the paper for research purposes? Do they have to keep separate clippings files, since the electronic version is unreliable, and has been purified in some way for later consumption? Is there an active policy under way here that should affect your Ethics statement? How are decisions made to ‘improve’ the content of articles that have already appeared in the printed edition? Why are these not considered ‘Corrections’ that would normally be posted in the relevant section? How often does this happen?”

I received a prompt response, but it was all very dismissive and casual:

“The change you noticed was simply the result of normal editing, which takes place constantly for news stories, both between print editions and for successive online versions. In this case, additional information (including crowd estimates) was added to the story between the early print edition and the final print edition, which meant something had to be cut for the story to fit in the same space. In most cases, the final print version is the one that remains permanently on nytimes.com, though in some cases a story continues to be updated or revised online even after the final print edition.”

So I countered as follows:

“But I must state that I think that you (and I am not sure who ‘you’ are in this case) are being far too casual about this policy, simply treating the process as ‘normal editing’. Is there an audit trail? Do you keep all versions? What changes are allowed to be made after the final print version? Why cannot the on-line version (which has no size constraints) include all the text? Is there any period of limitation after which no further amendments can be made? How do you plan to explain this policy to readers, whose ‘trust’ you say you value so much?

I am sure you must be aware of the current debate that is being carried on in the world of academic research, where annotations to URLs in serious articles often turn out to be dead links instead of reliable sources. A Times ‘page’ no longer has a unique and durable identity, which I believe is an important issue.

I look forward to some deeper explanation of this policy in the newspaper.”

Well, maybe I should get out more. As Sylvia would suggest to me: “You clearly need something better to do.”  But I maintain that it is an important problem, not just concerning journalistic integrity, and getting the story right the first time, and not correcting quotations that the speaker wanted to withdraw (which we are told goes on).  It is more to do with what is known as ‘content drift’ and ‘reference rot’. As Jill Lepore’s article states: “. . .a 2013 survey of law- and policy-related publications found that, at the end of six years, nearly fifty per cent of the URLs cited in those publications no longer worked. According to a 2014 study conducted at Harvard Law School, ‘more than 70% of the URLs within the Harvard Law Review and other journals, and 50% of the URLs within United States Supreme Court opinions, do not link to the original cited information.” A more subtle problem is that the links may work, but the content may have changed  ̶  may have been edited, corrected, improved, revised, or sanitised. For researchers like me, this can be very annoying, as books these days frequently cite URLs rather than printed sources in their references, and when those pages do not exist, one feels cheated, and may also wonder whether they have been modified. The academic process has been debased. If one has text in the New York Times that is no longer on the archive, does it still exist? Is it still valid? Do I really have to maintain my clippings files, as opposed to an index of URLs? (To make her point, the Times Vice-President had to send me a scan of the two printed versions of the relevant page in question.)

We shall see. I haven’t received a follow-up to my second inquiry yet. Either the Times doesn’t believe it is an issue, or the managers there are having a big debate about the topic, which they don’t currently wish to share. I’ll provide an update if I do hear anything.

The normal set of Commonplace Updates this month. (February 28, 2015)

Leave a Comment

Filed under Media, Personal, Technology