Category Archives: Technology

The Mystery of the Undetected Radios (Part 4)

“A masterpiece of Radio Precision”? see below

News Update:

Alert readers will have noticed that I received important communications from Roland Philipps (the biographer of Donald Maclean) and from Jan-Willem van den Braak (the biographer of the Abwehr spy Jan Willem ter Braak), whose work is being translated from the Dutch for publication in the UK. I shall report on the outcomes of these dialogues in next month’s report.

An observation on Guy Liddell and Roger Hollis by one of my contacts in intelligence inspired me to break out in verse on the subject of MI5’s efforts to counter Soviet influences. The doggerel can be found at DiaryofaCounterEspionageOfficer.

************************************************************************

After I had put Part 3 of this saga to bed at the end of September, some thoughts that I had vaguely touched on in earlier episodes returned to me with more vigour: What if the mistakes over ter Braak and the controversial report by Walter Gill (which effectively concluded that domestic wireless interception was not necessary) were both deliberate exercises by MI5 and its partners? Were the plans for the double-cross operation that far advanced in the last few months of 1940 that it was considered vital to give indications – in the belief that the Abwehr would pick them up – that Britain’s wireless interception policies were so weak that German agents could essentially roam at will, and broadcast home undetected? After all, as early as September 1939, Guy Liddell of MI5 had written that ‘it was in our interests that the Germans should regard us as grossly inefficient in these matters’, and that ‘if they thought our organisation was good they might well ask how it was we managed to get his [SNOW’s] messages through’. And were the Abwehr’s planting of obviously fake identification cards on its agents a deliberate ruse to determine how gullible the British counter-espionage services were?

These may be utterly fanciful notions, but they have a modicum of sense about them, as all such exploits at face value are very difficult to explain. One has to assume that agencies like MI5 and the Abwehr were continually thinking: how will our enemy counterpart think and act? (A British FOES committee did in fact exist: Guy Liddell described it as ‘an inter-services committee that tries to put itself in the position of the enemy intelligence service’.)  And, if some sensible insight were applied, each intelligence section should have assumed that its counterpart, because of native influences, might in some circumstances act in a different fashion. Thus, in this instalment, I start to explore the variations in the strategies and successes of the major European-based espionage/sabotage organisations: SOE (Special Operations Executive), the German Abwehr, and the network of the Soviet Union’s GRU and KGB spies, and what their controllers should have learned from their experiences in one theatre of war to apply to another. There is a symmetry in some of the things undertaken by each organisation, as they strain to develop measures to confound the forces trying to counter them. Yet one can also spot asymmetrical aspects, driven by the idiosyncratic nature of each force, including their overall motivations and objectives, the personnel they selected, the territorial dimensions, and the cultural drivers behind their operations. It is hard not to suppose, however, that the policies of each were not somehow affected by their knowledge of what their adversaries were doing with their own offensive activities.

The focus of my research in this series has been the detection of illicit wireless. It is worth recording here that the primary purpose of what is commonly known as RDF (Radio Direction-Finding, but implicitly including Location-Finding) had, before the war, been the interception and decryption of government (e.g. military, diplomatic and police) traffic. Initially, precise location was not as important as content. As countries started to perform intelligent traffic analysis, however, the origin – and mobility – of transmitting stations, especially military units, became much more significant, often providing intelligence even though the underlying messages could not be decrypted. Then, as the combat started, organisations had to start to apply their knowledge to the possible threat of illicit stations operating behind their own lines.

With all three combatants, the techniques for long-range triangulation were well-developed by the time war broke out, and thus could in principle be quickly adapted for identifying illicit domestic transmissions. The paradox was that, owing to the vagaries of the behavior of radio waves, it was often easier to pick up transmissions originating abroad than those issuing from inside the country’s boundaries. As I explained in Part 1 of this saga, low-powered wireless sets operating on high-frequencies in domestic territory, designed to exploit ‘bouncing’ off the ionosphere, were often hard to detect because of the skip zones involved, and widely dispersed human interceptors would have been needed to pick up their ground waves. Such a set-up was possible in the United Kingdom, but not in the expanding German Reich. Moreover, the finer granularity required for locating individual wireless sets (at building-block or house level) demanded new mobile equipment and techniques not explored in long-range location-finding.

As I discuss the strategies and challenges of the three espionage forces, and attempt to assess their effectiveness, I shall be considering them under the following criteria:

  1. Operational leadership: How good were the directors in planning how objectives should be met, and following up by providing the motivation, material, and structure to allow agents to be successful?
  2. Quality of operators: Were agents with the appropriate profile chosen for the job in hand?
  3. Quality of training: Did the agents receive thorough and suitable training?
  4. Quality of equipment: How effective was the equipment (primarily wireless apparatus) for the location of operation and for transmission needs? Were conditions such as local power supply properly taken into account?
  5. Operating procedures: Were safe and secure operating procedures defined, and did the agents follow them?
  6. Remote support: Did the agents receive reliable and effective support from their home controllers?
  7. Detection capabilities: How effective were the enemy’s radio-detection and direction-finding mechanisms?
  8. Social environment: How hostile or sympathetic was the social environment in which they had to work?
  9. Counter-Intelligence strategy: What goals drove the counter-espionage strategy of the enemy on whose territory the spying took place?

June 1941 constitutes the major chronological dividing-line in the conduct of wireless espionage. (In the light of my research, I have deviated from the temporal Phases identified in my first post in this series, which had Phase 1 completing at the end of 1940, and Phase 2 winding down in June 1942.) The Nazi invasion of the Soviet Union immediately changed the German attitude in Soviet counter-espionage from one of wary passivity to aggressive pursuit. The Russian stance in illicit communications switched from cautious dormancy to careless urgency. For Britain, it signalled that any planned invasion of the island nation had been postponed indefinitely: the timing coincided with the transfer of RSS to SIS, and the implementation of the new structure in MI5 under David Petrie. The date has less significance for SOE: it was still in an experimental, groping stage in the summer of 1941, with only two radio-stations established in France by that time. My analysis thus presses forward in this dimension of espionage and sabotage to address the continued struggles of the unit into 1942. I now summarise the activities of the three agencies in this period before delving into more detail.

I have shown how the greatest intensity of Nazi attempts to infiltrate British territory occurred in the autumn of 1940 (Operation LENA), with a couple of reconnaissance landings (by Jakobs and Richter) occurring in the spring of 1941 – i.e. before Germany’s alliance with the Soviet Union turned into a clash. By then, with the plan to invade the United Kingdom abandoned, and Hitler’s attention now directed to Operation Barbarossa, the agents whom the Abwehr had apparently successfully installed in Britain took on less importance. They appear to have been largely forgotten, or abandoned, and it took the arrival of new ‘spies’, such as TRICYCLE, GARBO and TREASURE (whom I shall cover in the next chapter), to re-activate the espionage – and the Double-Cross – project. Yet using wireless was not at the forefront of the Abwehr’s plans, and MI5, in their efforts to facilitate the passing on of fake information, had to be very careful and imaginative when encouraging use of the medium.

As far as Britain’s own plans for espionage and sabotage were concerned, Churchill had in the meantime (July 1940) established the SOE as a force to penetrate Nazi-occupied Europe, and to soften up and harass the invader’s government of occupied territories. Yet this was not primarily an espionage organisation, like SIS (whose network had been almost completely destroyed at the outset of war.) It was an outfit committed to sabotage, and, while wireless communication became a critical part of its operational infrastructure, the technology was used more to arrange for shipments, drop-offs, and pick-ups, and only secondarily as a mechanism for providing intelligence. Sabotage operations also drew more obvious attention from the enemy: furthermore, in the first two years of its existence (i.e. until the summer of 1942), SOE was hampered by being reliant on Section VIII of SIS for its wireless equipment, wavelengths, codes, etc. The experience in responding to illicit SOE transmissions in France may have given the German counter-espionage agencies a leg-up when the Soviet apparatus fired up in the summer of 1941, but, as will be shown, the evidence for this is shaky.

When Hitler invaded the Soviet Union, all Soviet agents in place in Germany were immediately activated to provide intelligence about Nazi war-plans. Yet they had not been completely dormant before then. The situation was in fact more complex than that. After the show-trials and purges of 1937-1938, the KGB and GRU networks had been patiently rebuilt – not just in Germany, but across most of Western Europe. As early as May 1940, however, when Paris fell, Moscow suspected that relations with Nazi Germany – despite the Ribbentrop-Molotov Pact – might deteriorate, and diplomatic representatives (e.g. Kobulov in Berlin) started building networks of informers, not only in Germany but also in Poland and Czechoslovakia. Elsewhere, the Soviet Union’s spies had long been active, such as in the origins of the famous Red Orchestra group in Switzerland, led by SONIA (Ursula Kuczynski) and DORA, the Hungarian Sándor Radó, who had been recruited in 1935, and moved to Switzerland in 1939. Before 1941, however, couriers, and communications through local Soviet embassies, had been a much more convenient method of passing information than the use of wireless transmission methods.

Abwehr Spies up to June 1941

Admiral Canaris of the Abwehr

The decision to infiltrate spies into Great Britain in late 1940 was taken at short notice, but, like many events of a time when feints and deceptions were part of the strategy, the exact date when Admiral Canaris initiated the LENA programme is uncertain. In 2018, Bernard O’Connor, relying on the rather dubious transcription of Lahousen’s War Diaries claimed by Wighton & Peis sixty years earlier, asserted that Canaris told his Abwehr officers as early as June 22 that gathering intelligence on Britain, in preparation for the planned invasion, was of the highest priority. That early preparation is vaguely echoed by Niklaus Ritter in his 1972 memoir, Deckname Dr. Rantzau, where he improbably describes being in the company of Caroli (SUMMER) and Schmidt (TATE), ready for their departure some time in July, when they had already completed their eight-weeks’ training. Yet Ritter’s memory was at fault: he describes them as leaving on the same plane – something which the British archives strongly refute, so one must question the reliability of his memory. John Lukacs, in The Duel, represents Admiral Raeder as still trying to talk Hitler out of invading Britain as late as July 11, with Hitler responding in terms of wanting to make peace with the United Kingdom. O’Connor and Ben Macintyre both refer to a conference held in Kiel ‘some time in July’ to plan the details of the LENA operation, an event confirmed by the Kew file on the Hamburg Abwehr officer Praetorius (KV 2/170-1), and given precision by KV 3/76, which sets it as taking place on July 16. That would dovetail with Ritter’s account that eight weeks of training had to be accomplished to meet Hitler’s deadline of September 15. 

Praetorius’s recollection was that the agents parachuted in at this time would ‘only have to be of independent means for 6-8 weeks as by at time the invasion of England was expected to be an accomplished fact.’ Yet the chronology does not work. If a decision had been made in July, the recruitment and training of agents was supposed to take eight weeks, and their subsequent independent existence on British soil might have been expected to take another six to eight weeks, the latest date for a successful invasion would have to be placed as late as early November. While Anthony Cave-Brown gave August 1 as the date that Hitler issued his Directive 17 to prepare for the invasion of Britain, Operation SEELÖWE (SEALION), Churchill himself reported it as being on July 16, with Hitler’s apparent objective of having his forces arrive four weeks later. On September 11, however, Hitler had to delay the invasion order until September 24, and on September 17 he ordered the indefinite adjournment of SEALION, and formerly cancelled it on October 12. Yet the first LENA agent, Caroli (SUMMER) did not parachute in until September 3, and his colleagues were still arriving in early November. It sounds as if Canaris gave Hitler unreasonably optimistic indications of the speed with which agents could be recruited and trained: if Hitler had been able to stick to his original plan, there would have been no planting of infiltrators in the United Kingdom, successful or not, to assist the invasion. Yet the program unaccountably went on after invasion plans were suspended, which would have made nonsense of the ability of the agents to survive independently for a few weeks.

Given the haste by which recruits had to be selected, vetted, and prepared, it is thus difficult to take seriously the claim made a few years ago (in Monika Siedentopf’s Unternehmen Seelöwe) that the invasion of Britain was sabotaged by Canaris and his team, in that they selected unsuitable candidates as spies who simply let the side down. Apart from the chronological problems listed above, however successful the few who landed might have been in evading capture, their effect on a planned invasion that required destroying the Royal Air Force would have been minimal either way. But that does not mean that the Abwehr’s project was not quixotic, or even cruel. The agents were chosen in a hurry: they were not native Germans, but mostly citizens of bordering countries (Denmark, Sweden, the Sudetenland – the last, of course, transferred from Czechoslovakia to the German Empire). Some were diehard Nazis, some were lukewarm, others were pressured into signing up by threats. The belief was that agents from outlying countries would fade into the background more easily than native Germans: some had spent time in the UK beforehand, but, overall, they were hopelessly unprepared for life in the United Kingdom. And as potential observers, they were untrained. Reports at Kew indicate that ‘though they were expected to report on such military objectives as aerodromes, land mines and gun batteries, on examination they showed only a vague idea of the significant points to note.’  They had ‘only an amateur knowledge of transmission technique.’

The main point, however, was that the spies of the LENA operation were not expected to be operational for long, a fact that is reinforced by the way that most of them were equipped. More than half of the eighteen (the exact number is debatable) who landed, either by parachute or boat, between September 3 and November 3, 1940 either carried with them a transmitter only, or no wireless equipment at all. A transmitter might have been useful for sending a brief set of dazzling reports about air defences, bomb damage, or weather conditions, but without an ability to have confirmed whether one’s messages were being received correctly, it would have been a short and demoralizing career. For those agents being parachuted in, wireless apparatus was a significant health hazard: at least two spies were injured by virtue of their collision with the earth when harnessed to sets weighing twenty pounds or more. Most had not practiced a parachute-jump before. Moreover, many were told in Hamburg that there was not enough shock-proof material available, and thus they would be equipped with transmitters only. If wireless sets were dropped separately, there was the risk of the apparatus’s never being found. TATE demanded he be equipped with a combined Transmitter/Receiver. As his Kew file reports: “His controller, RITTER [Captain Rantzau] then informed him that arrangements were being made for him to take with him to England a separate transmitter and receiver and also a large transmitter (called a ‘Z.B.V.’) which would be dropped separately and which he could destroy if the smaller sets were unbroken after landing.”

MI5’s analysis of the equipment the agents were provided with would indicate that they did not have a high chance of success in trying to contact their controllers. The boat agents (Meier, Waldberg, Kieboom and Pons, who arrived on the Kent coast) were equipped with compact and light cases, one weighing 7 lb., and containing batteries and connecting wires, the other weighing only 4 lb., containing the transmitter, aerial and spare valve. (This was in dramatic contrast to the bulky devices that SOE agents were required to take to France or, say, Yugoslavia, in following years.) Yet the experts judged that such low-powered devices ‘would require exceptional conditions to work over 100 miles’, with an expected range of nearer 50 miles. *  If that judgment is correct, it would show an extraordinary misjudgment by the Abwehr experts: reducing power to such a degree that transmissions would not only be undetectable locally, but would also not have enough energy to reach their intended target. This statistic is put into perspective by the fact that the distance between the port of Southampton and Cherbourg is over 100 miles, while German wireless agents were transmitting home from as far afield as New York and Brazil.

[* This opinion needs to be balanced against that of E. H. Cookridge, who, in his 1947 work Secrets of the British Secret Service, described Kieboom’s equipment as ‘a masterpiece of radio precision’, following up by claiming that ‘the transmitter allowed to send [sic] messages over a range of more than 600 miles, yet was so small that it could be hidden in two leather boxes  . . .’ (see Figure below). In his Preface, Cookridge thanked the Foreign Office, the War Office, the Home Office and the Lord Justice’s Office for their assistance, so his book should probably be regarded as an item of selective disclosure for propaganda purposes, perhaps maximizing the wireless threat.]

SNOW’s transmitter was reported to have a much more realistic range, of up to 1200 miles.  Likewise, CAROLI’s (SUMMER’s) equipment was much heavier and more powerful, but would have a corresponding disadvantage of requiring much more space to set up the aerial. “Aerials provided would not be easily untangled and satisfactorily erected except in secure privacy with plenty of space. E.g. indoor space 60 ft. long or a secluded wood with a fairly clear space 6o ft. long with trees etc. on which to tie the end of the aerial to a height of at least 6 ft.” How a spy in tight wartime conditions, in densely populated England, was supposed to accomplish such a task is not clear. A tentative conclusion by the report at KV 3/76 was that the agents were so ill-prepared that they should perhaps be considered as decoys.

Kieboom’s equipment details (from Cookridge)

Nevertheless, it seems that the Abwehr stations stayed observant, looking for transmissions from the agents. The same file, K 3/76, based on interrogations of the six prominent spies captured by September 1940, supplemented no doubt by RSS interception and decryption of Abwehr exchanges, discloses the following: “It appears from other sources [sic: surely a code for Ultra decrypts] that a constant watch is kept by Hamburg, Berlin, Paris and Cherbourg, for the reception of any wireless messages by all agents despatched to the U.K.  This is presumably in order to make sure that messages shall not be missed through bad atmospheric conditions.” The advantage gained by the German Reich’s territorial extension into Northern France (which also aided triangulation for location-detection) was counterbalanced by the fact that ENIGMA radio communications had to be used rather than highly secure land-lines, which allowed British Intelligence to tap into the plans and processes of the Abwehr. Moreover, by this time, Hamburg (which would have had secure contact with Berlin) was shifting its attention to Norway, placing the responsibility for Britain on to Paris and Cherbourg. A dangerous increase in interceptible traffic was caused by the fact that the Abwehrstelle in Brussels was used as an intermediary point for traffic, with messages passed to it from advance stations to be decrypted, and then passed on to Hamburg, Paris, or Berlin.

Because nearly all of the spies were picked up soon after they landed, little can be said about the adequacy of their training. Ter Braak apparently struggled with his receiver: concealing aerials in densely-populated Britain, with vigilant landlords and ladies, would have been a problem. TATE had only one frequency to work on, which was effective only in daylight hours: this inhibited his activity later. TATE admitted that he had been taught the fundamentals of operating, but nothing about wireless theory, which would mean he would be helpless when problems occurred. He said that he only knew “the practical details of how to join it up, erect the aerial, and tune the transmitter by the lamp. He thought he could spot a disconnected wire inside, but that was about all”. As Reed of B1A reported: “He had been instructed to join motor-cycle batteries in series, but three 6 volt batteries would burn out his valves.” Consequently, even with MI5 assistance, TATE struggled to make consistent contact. Reed reported, on October 1, that ‘experiments with [TATE’s] wireless were unsuccessful due to inefficiency of aerial provided with a set of so small an output.’ His first successful message was not sent until October 10: he was supposed to send a postcard in invisible ink to a contact in Lisbon if his wireless failed to work. She never received the postcard.

TATE had quickly understood that his life depended upon abandoning his Nazi affiliations, and following the instructions of his new captors. Unlike SUMMER, he did not have second thoughts, and thus did not employ any security code to indicate that he had been turned. (He claimed that the possibility of being captured and used had never been acknowledged by his trainers, and he thus did not have such a code.) He initially operated his set himself, and thus displayed a consistent ‘fist’. Yet the overall message to be gained from this exercise is that the Abwehr controllers soon lost interest. As early as September 7, Field-Marshal Jodl told the Abwehr to open up operations against the Soviet Union. The realization that German could not dominate the skies above Britain, and that a winter invasion across the Channel would simply be a recipe for failure, had by then convinced Hitler that it was time to turn his attention to the East.

What TATE’s files at the National Archives show is the enormous lengths to which MI5 and RSS went to experiment with his apparatus, attempting to make contact with Wohldorf. While SUMMER’s set had been shown to work quite quickly, MI5 provided their counterparts at RSS with all the details of call-signs, frequencies, and times so that the location-finding network of interception towers at Thurso, St Erth, Gilnakirk, Sandridge, Cupar and Bridgewater could gauge the strength of the signal, and give back advice. Hughes (W6B) and then Reed (who was on secondment from the BBC) had to move the set around from city to countryside, change the length of the aerial and fine-tune its alignment, and also have the complex instructions for TATE’s back-up set translated before they were able to send transmissions of consistent quality. Yet they were already sensitized to the need to avoid German direction-finding – to a degree that was unnecessarily cautious: they believed that the transmissions could have been localized to an actual building (e.g. Latchmere House), a degree of accuracy way beyond what the Funkabwehr was capable of at that time.

Meanwhile, agent SNOW (Arthur Owens) was being kept in close confinement. It should not be forgotten that SNOW was the original Abwehr agent equipped with wireless, and was notionally active right up until April 1941. Yet the first experiments with wireless were haphazard: he was supplied with a clumsy and reliable transmitter (only) in February 1939, but, since he was able to meet his handler, Ritter, in Hamburg until war broke out, and, after that, arrange regular rendezvous in the Netherlands and in Belgium until the Nazis overran those countries in May 1940, the use of wireless to pass on intelligence was not so critical. Of course, that made the task of monitoring what he said impossible, and suggestions that SNOW had betrayed his country by revealing suitable targets for bombing (i.e. going beyond the ‘chickenfeed’ that he passed in his encrypted messages) caused MI5 to terminate him, and incarcerate him for the remainder of the war.

Agent SNOW

MI5 was aware of SNOW’s wireless usage from the day his set was picked up. SIS even broke the set, and had to repair it. But SNOW did not make his first successful transmission until late August 1939: soon afterwards, MI5, aided by his wife’s jealous reporting of his duplicitous activity, arrested him, and then found both his transmitter, and then a receiver, concealed at his property in Surbiton. Under MI5’s tutelage, SNOW moved house to premises where his aerial would not stand out so obviously, and transmitted regularly on weather and less than critical military operations and preparation. The first Double-Cross message was sent on September 9, but no confirmation of receipt occurred for some weeks. At some stage in October, Maurice Burton, who had earlier checked to verify that SNOW was transmitting as instructed, took over the operation of the apparatus, and eventually a new afu transmitter-receiver was delivered through a third party.

Whether the Abwehr had been careful enough to pay attention to SNOW’s radio ‘fist’, or whether Burton was adept enough to emulate it, is not clear. The archival reports give every indication that Robertson and his team assumed that Ritter must have concluded that SNOW was being controlled by MI5. Guy Liddell even wrote, on February 2, 1941: “Another point that occurs to me us that the Germans must now be wise to the game of collaring an agent and forcing him to use his wireless set in our interests. There is in fact evidence that they are doing it themselves.” Yet the Abwehr used what SNOW fed to them concerning passports and ration cards to supply the LENA agents, and lure them to their doom or glory. Exactly who was deluding whom by the time SNOW was regarded as a high security risk may well never be established. A triple agent works only for himself, trying desperately to play one employer against the other in order to survive. Interrogators of Ritter after the war concluded that he had realized that SNOW had been turned, but, when Ritter wrote his memoir in 1972, he gave no suggestion that SNOW was anything but the genuine article. Ritter believed that SNOW was being used by MI5, but that the Abwehr had outwitted them. He certainly would not wanted to have admitted to his bosses in Berlin at the time that he had been deluded. Other Abwehr officers interrogated were more outspoken and direct about their suspicions: I shall explore these in a later chapter.

MI5 and RSS gained much from these experiences. They learned about the enemy’s equipment, and the RSS was able to test out its interception and location-finding techniques when they applied their sensors to TATE’s transmissions, in order to evaluate how effective they were. Yet this was a precarious time for MI5: the seeds of the successful XX Operation were quickly sown, but Liddell and others also came to realise that allowing ‘undetected’ radios to operate would require the existence of a ham-handed and inefficient detection service for them to evade interception. This concern would continue to dog MI5 throughout the war –  the fear that the Germans must assume that the wily British had better radio-detection finding equipment than appeared to be the case, and would thus assume that their agents were not operating freely. And, as I pointed out in my article on ter Braak, is it not somewhat ridiculous to think that, in densely-populated Britain, with a citizenship well advised to look out for suspicious activity, that an obvious foreigner, with accented English, could traipse round the country picking up information, and then return to some lodging where he managed to conceal the existence of a lengthy aerial while sending in his reports?

For the Abwehr, their LENA spies were dispensable. The espionage service did not think they would survive long, and it had low expectations of their deliverables. As a July 1944 report submitted jointly by MI5 and SIS declared: “According to the calculations of one Abwehr officer, eight-five per cent of the agents dispatched were never heard of again; ten per cent turned in information which was either worthless or false; the remaining five per cent provided sufficient accurate reports to justify the expense of the remainder. The first two clauses of this sentence may have a greater validity than the last.” (The last observation was perhaps a tacit hint of the XX Operation.)  Agent Richter may have been sent in to verify whether TATE had been turned, but the fact that the Abwehr never learned anything from Richter did not deter them. The Abwehr no doubt had it confirmed for them how difficult it was to infiltrate an island nation. MI5, even at that time, took pains to ensure that manipulated transmissions took place in locations where the spy was supposed to be, but the state of the technology on the German side at that time was probably inferior to that of the British: even with appropriate triangulation, transmitters could not be ‘pinpointed’ to much less than a circle of 20-mile radius, and there is no evidence that the Germans bothered. Yet the awareness of RDF as a technique for counter-espionage would have registered with them, and would come sharply into focus a few months later.

As a coda, and a point to be picked up later, the British apparently recognized, after the war, the Germans’ superior techniques in detection and direction-finding. In his 2011 memoir of his days at Bletchley Park, Secret Days, Asa Briggs writes that GCHQ acquired a field north of Bletchley that was later named Furzton. “A radio direction finding system developed by the Germans was installed there. Judged superior to all existing British systems, it consisted of an outer circle of forty and an inner circle of thirty smaller metal masts,” he adds. Yet a search on ‘Furzton’ fails to come up with anything else. (Google led me to Hinsley’s and Tripp’s Codebreakers, a book I own, but with no incidence of ‘Furzton’, which does not appear in the Index.) To learn more, perhaps, we must wait for the Official History of GCHQ to appear next year. The overarching conclusion must be that, after the initial excitement in setting up W Division in MI5 in August to track illicit wireless, the transfer of RSS to SIS, and the establishment of the XX Operation, accompanied by the belief that all German agents had been turned, incarcerated or executed, concern about  illicit radio transmissions, whether they came from foreign embassies, maverick civilians, Soviet spies, or even undetected German infiltrators, the demand for prosecution of such activity through urgent and efficient location-finding went somewhat off the boil.

The Funkabwehr

The Nazis had their equivalent of Britain’s Radio Security Service, the Funkabwehr, sometimes translated as the Radio Defence Corps. Yet the Germans came rather later to recognize that the threat of domestic illicit wireless communications required a more committed function. Created by Hans Kopp in 1940, the Funkabwehr reported to the OKW, the Oberkommando der Wehrmacht, and readers may find references to the OKW/WNV/FU, a typically precise but wordy example of how the Germans described their units, Wehrmacht Nachrichten Verbindungen Funküberwachung, loosely the surveillance of radio intelligence and communications. Unfortunately, a good history of the Funkawehr remains to be written, as German records are unavailable. For a detailed history of the organisation, the Wikipedia entry at https://en.wikipedia.org/wiki/Funkabwehr is reasonably solid, but has a very shaky chronology, is written too much in the passive voice, and in my judgment contains several errors. * Moreover, it is highly dependent on a 1946 report compiled by the RSS itself, which can be seen at https://drive.google.com/drive/folders/0B_oIJbGCCNYeMGUxNzk0NWQtNzNhZi00YWVjLWI1NmItMzc2YWZiZGNjNjQ5, a folder in Christos T.’s excellent website dedicated to military intelligence matters. While this account lacks the benefit of historical distancing, and integration of much new material, I shall not repeat here the detailed evolution of the Funkabwehr’s capabilities.

[* The danger of referring to Wikipedia, or indeed any on-line source, is that the entry may change suddenly, or even disappear. The Wikipedia entry on the Funkabwehr has been expanded considerably since I started this article.]

Germany and Great Britain had long maintained ‘Y’ (signals interception) capabilities, the focus of which had been primarily diplomatic and political communications of foreign powers, but assumed interest in military plans and operations as war approached. Britain had listening posts throughout the empire, and Germany had established a similar network within the German borders. The Nazi interest in the years before the war appears to have been directed more against the Soviet Union: by 1937, from their intercept stations at Treunbritzen, Jüterbog, Königsberg and Breslau, they were picking up a large amount of NKVD traffic stretching from Murmansk to Odessa. This activity no doubt continued during the period of the Nazi-Soviet pact (August 1939to June 1941), and helped Hitler prepare for operation Barbarossa.

German Communications (from RSS report)

Yet, as the awareness of possible clandestine wireless activity within each nation’s borders increased, approaches to the problem started to diverge. True, the general methodology and use of technology was very similar, but the geographical and political constrains led the adversaries down different paths. First, the borders in the European theatre of operations remained stable for the British: the Germans had to deal with their fast expanding occupation of new territory. While it provided for a steady increase in suitable locations for interception stations (e.g. Brest, in France), it also increased the possible quantity of subversive communications. It also put more strain on inter-unit communications, since secure landlines were no longer available, and thus exposed more secret information transfer to interception itself. Moreover, the operations were frequently taking place in environments hostile to the invaders, with the risk of sabotage, and, certainly, non-cooperation.

Another aspect was duplication of effort. It sometimes comes a surprise to learn how fragmented the approach of a totalitarian nation could be to intelligence matters. Hitler encouraged rivalries, however, and there was a large absence of trust between organisations. In fact, the function of the Funkabwehr was split between the OKW unit and a section of the Ordnungspolizei (or Orpo) called the Funkabwehrdienst, which was under the control of Heinrich Himmler. Both units were responsible for the location and apprehension of those transmitting illicitly, but for most of the war their missions were divided by what could seem to be an absurd and unproductive distinction. Orpo was responsible for identifying clandestine operations against the government and the regime, while the WNV/FU directed its efforts against activities against the state. How they could confidently conclude which category a transmission belonged to before analysis, or why they discounted the fact that some factions might effectively be fighting both, has not been explained. Britain, on the other hand, maintained a unified control over interception, and generally benefitted from the large amount of trust that existed between the military, the political, the interception and the cryptographic organisations. It was not until 1943 that the Orpo and the WNV divided their tasks more sensibly along geographic lines.

One critical matter that the RSS report brings to the surface is that of distortion of signals, and how the proximity of electrically conductive objects of dimensions close to the length of the wave could affect both reception and interception. What the receivers of transmissions initiated from agents in enemy territory were interested in was content, and weakening of the signal would affect successful reception. Communication was one-to-one: the receiving station would be the sole unit dedicated to trying to capture a transmission. Distortion could mean that the signal was lost completely, or fell into the skip zone. Location was not important to such receivers: indeed, transmitters were encouraged to move around (with those clumsy antennas – but not too far afield so as to jeopardise the signals plan) to evade detection. Interceptors, on the other hand, were rarely interested in content: they probably did not have the resources or time to decrypt the messages. What drove them was location, so that they could quickly eliminate (or turn) the offending agent and equipment. Distortion might not mean complete loss, as multiple detectors had to be in place to perform the triangulation necessary, but it could mean that a faulty indication of location was reached.  

Yet it was all a hazardous business. The presence of interfering objects (buildings, mountains), by radiating signals in new directions, can confuse the process of triangulation, or cause the assumed location to be challengingly large. This distortion can also occur simply because of the erratic behavior of the ionosphere, especially at time of sunrise and sunset. Guy Liddell reported, on February 10, 1941 that ‘the alleged parachutist’s [JAKOBS’s] transmitter from this country was heard again on Sunday but turned out to be a communication between Paris and Cracow’. In a 1944 report, written by British Intelligence to prepare its officers for the invasion of Europe, appears the following observation: “The skip distance of any transmitter is calculable in normal circumstances; but, occasionally, owing to temporary changes in the atmosphere freak results may be obtained, as in the summer of last year when the short wave transmissions of Chicago police cars were clearly (and tiresomely) audible on the south coast of England.” (I am confident that this pamphlet, available at Kew at WO-279-499, was written by Hugh Trevor-Roper: he was the Abwehr expert, and the prose has a donnish flair, and is regularly sprinkled with Latin phrases.) We should also remember that Britain’s scheme of catching all groundwaves by the dispersion of interceptors throughout the country could not conceivably be mirrored in Germany, let alone in its expanded territories. The dynamics of the cat-and-mouse game played between spies and enforcers must be evaluated in this context.

Overall, therefore, the reputation of German counter-intelligence as a ruthless and efficient machine, which has been encouraged by war-movies, and even historians of SOE, is certainly overstated. The Funkabwehr suffered from duplication, tensions of centralisation and decentralisation, inadequate training, poor communications, a shortage of qualified amateurs (unlike Britain’s Voluntary Interceptors), too rapid job movement, insufficient mobile units, sometimes poor quality equipment, and lack of appropriate language skills. Coordinates provided by remote RDF were frequently too vague to ensure successful local house-hunting. Certainly the discovery of the Soviet Rote Kapelle spy network in the summer of 1941 moved operations into a higher gear, but the organisation in France (for instance) remained weak until as late as 1943. The RSS report assesses the technical resources at the outbreak of the war as being ‘completely insufficient’, given the rapidly occurring military victories and the increase in occupied territory’. It tells a story of frequent failure, that it took weeks or even months before a transmitter was at all precisely located. Yet the RSS seemed also to be under the impression that the number of Allied W/T agents was rapidly growing in 1940, an illusion that is undermined by the histories of SOE that have appeared. The more innovative technologies and approaches of the Funkabwehr thus occur well after the period under the microscope in this chapter, and will be analysed in a future episode.

SOE and Wireless: 1940-1942

The SIS organisation in Europe had been greatly weakened by the beginning of war, and the Venlo incident on November 9, 1939 (whereby the Abwehr captured SIS officers in Holland, and gained detailed information about the service’s structures and personnel) crushed it. SOE was launched, with a charter written by the dying conservative Neville Chamberlain, and under the ministerial direction of the socialist Hugh Dalton, in July 1940. Its mission was to perform subversion and sabotage in those countries of Europe controlled by the Nazis. While Chamberlain declared that its operations should be tightly woven in to the greater military strategy of the war, this facet of its decision-making was never really clear. Was it supposed to disrupt the Germans’ efforts to produce war material? Was it designed to initiate minor diversionary attacks that would draw a high degree of military and police resources away from other arenas? Or was it intended to help prepare for the eventual invasion by softening up targets, and impeding troop movements? All these goals were troubled by the fear of what reprisals the Nazis might take on such incendiary activity, and what effect that might have on local morale. Moreover, SOE was always competing for resources – especially for aeroplanes and wireless equipment – and those often unfulfilled demands, hampered by other departments that questioned SOE’s effectiveness, meant that SOE had a very chequered history in the first two years of its existence.

The sources on SOE are fragmented. M. R. D. Foot’s SOE in France, originally written in 1966, and reissued in 2004, is an ‘official’ history, part of the Government Official History Series, but, as is clear from its title, covers France only. (In an interesting sidenote, Foot himself, in his 1976 work, Resistance, refers to SOE in France as a ‘quasi-official’ history.) Foot wrote another volume covering all of SOE, SOE: The Special Operations Executive 1940-1946, in 1984, but it is not an ‘official’ or even ‘authorised’ history. Its chronology is hazy, and it provides little detail on wireless equipment and procedures. After the war, an internal history was commissioned from an Oxford don, W. J. M. Mackenzie (who had not been employed by SOE), and was eventually published, in 2000, as The Secret History of SOE: The Special Operations Executive 1940-1945. In all three books, the coverage of wireless is very sketchy until 1943, after SOE’s own research and manufacturing facilities had been set up, and Colonel Gubbins rather belatedly introduced more rigorous signals procedures. Various memoirs refer to the use of wireless, but they are not always reliable.  A number of files have been released to the National Archives in recent years, but few records of SOE’s activities in the early years appear to have survived fire, destruction or the weeders, and what have endured are (so far as I can judge) all undigitised

This report focusses on SOE in France, as it was the earliest field of operation, and it is here that the most pressing lessons of wireless usage were learned. SOE had two units working in France: the F Section, which was run as a British operation, and the RF section, which was a Gaullist unit for which French nationals only could work. F thus depended mainly on agents of Anglo-French nationality who spoke the language fluently.  And it took many months before SOE sorted out is mission, recruited and trained people, overcame political opposition, and were able to start placing agents deep inside France. It had infiltrated a few agents equipped with wireless by sea, but their communications were apparently spotty. The first confirmed F agent to be parachuted in with a wireless set was Georges Bégué (aka George Noble), who arrived in unoccupied central France on the night of 5/6 May 1941.

It might be expected that the local populace would be more supportive of parachutists sent in to hinder and harass the invader, but it was not necessarily so. Up until Barbarossa, the French communist party had welcomed the Nazi allies of Moscow, and rapidly had to change their stance after June 1941. Before then, however, communists were a threat to subversive activities as possible informers. Even in Vichy France, considered to be safer territory, many peasants were loyal to the administration, and would betray illicit movements to the authorities, and hence to the Germans. SOE’s policy with wireless operators was open to criticism: it would send in a team of three (agent, courier, and wireless operator) rather than devolving the task of transmission and receiving to the agent him- or her-self.  Frequently the operator spoke no French, and might be idle for weeks at a time, which meant concealment and exposure were a constant concern. Yet progress was slow. Lorain (see below) writes that there were only two clandestine stations working in France for Section F in May 1941, and a year later, still only seven.

Thus one has to treat Foot’s claims about the rapidity with which the Germans developed direction-finding techniques with some skepticism. He reports that ‘the German wireless interception service had detected Bégué’s transmissions almost at once, had begun to jam them within half a week.’ The Vichy police was involved, and ‘D/F vans joined in the search’. Elsewhere, in a general commentary, Foot writes: “The German intelligence service’s wireless direction-finding (D/F) teams were numerous and efficient, probably better than the British, for whom Langelaan [George Langelaan, Knights of the Floating Silk, p 220] claimed that if ever an unidentified transmitter was heard ‘in a manner of minutes a first, rough direction-finding operation had been accomplished.’” Again citing Langelaan, Foot then goes on to make the following rather nonsensical observation: “If the transmitter was anywhere in the United Kingdom, in less than an hour experts equipped with mobile listening and measuring instruments were converging on the region where it had been located.” Why an official historian like Foot would rely on Langelaan as a source, when the author was an SOE agent who probably received the information second- or third-hand, is not clear. (Admittedly, Foot would not have been able to find reliable information in the archives, but that is no excuse for such slipshod reporting.) From other accounts (such as Liddell’s Diaries), it is quite clear that, during this period, the approach by RSS to suspicious signals was much less rigorous.

As for what the capabilities of the Nazi teams were, ‘converging’ might mean location-finding rather than physical movement, but the proximity of Augsburg and Nuremberg to each other [see below] would mean any attempt at triangulation with Brest on sites in Britain would be a very haphazard, as well as pointless, exercise.  Nevertheless, Foot goes on to write: “French operators in the field early discovered that a long transmission in a large town would probably bring a detection van to the door within thirty minutes. The Germans soon worked out a technique for establishing what part of a town a clandestine operator was working in, by cutting off the current sub-district and noting when the clandestine transmission was interrupted; then they would concentrate their efforts on the sub-district affected, and hope to track down quickly at least the block, if not the building, the set was working from.”

In his general book about SOE, Foot reinforces the message. “In towns, sensible organisers and wireless operators took care not to see too much of each other; for the wireless operator was always the circuit’s weakest point. The Germans, like the British, kept a constant watch on every wireless wavelength, and it took only twenty or thirty minutes for a team of their armed direction-finders to get within a few yards of an operator who was fool enough to remain on the air so long. Relays of thirty clerks with cathode-ray tubes in the Gestapo’s headquarters in the Avenue Foch in Paris, for example, kept up a continuous watch on every conceivable frequency. When a new set opened up, it was bound to show up on a tube; the frequency could be read off at once. In a couple of minutes, alerted by telephone, direction-finders at Brest, Augsburg and Nuremberg were starting to take cross-bearings; within a quarter of an hour, detector vans would be closing in on the triangle a few miles across that the cross-bearings had indicated. Some of SOE’s early organisers in France and Belgium insisted on sending messages so verbose that their operators had to remain at their morse keys for hours at a time; and, inevitably, they were caught.

German Position-Finding, Phase 1 (1942?) (reproduced from Pierre Lorain’s ‘Secret Warfare’)

It did not take long for Gubbins, as head of operations, to spot what was wrong, or for the signals training school at Thame Park to start to impress on operators – as Beaulieu explained to organisers – that mortal danger lay in trying to send long messages by wireless.”

Yet all this is undated, and perhaps an indication why this analyst is wary is that Foot immediately follows this last passage with the following: “By the winter of 1943-4 – hardly before time – there was an order: no wireless telegraphy (W/T) transmission was to last longer than five minutes.” In the context of the war, this is an enormous chronological jump. Foot lists several other operations (Forman and Labit, DASTARD, Bloch) in the second half of 1941 that he claims were terminated because the operators stayed on the air too long, and were trapped by the efficiency of German detection-finding. Yet it is perhaps more likely that many of these agents were betrayed by sloppy tradecraft, or visible behavior that prompted the interest of citizens who felt it their duty to report such activity before they were arrested for ignoring it. In fact Mackenzie tells us that Labit (the wireless operator) had to escape to the Unoccupied Zone without his set, while his partner Cartigny was probably shot. Some gave the game away by weak identity cards, or obviously wrong serial numbers on notes, the same types of error that had bedevilled the LENA spies. In Resistance, Foot undermines his argument by writing: “Early in the war, the Germans worked the process [of interception] clumsily, but by the spring of 1943 they had main intercepting stations in Augsburg, Berlin, Brest, Nuremberg, and no doubt elsewhere.” Again, a distressing lack of precision, and a big chronological leap.

In his largely pictorial study of the use of wireless in the French Resistance, The Clandestine Radio Operators in France (2011), Jean-Louis Perquin presents an arresting account of the German special unit ‘dedicated to the detection of clandestine emissions’, describing a complex web connected to three detection-finding centres located in Brest, Augsburg and Nuremberg, and backed up goniometer trucks with equipped with the latest technology. Yet, again, chronology is vague: the text indicates that the procedure described was deployed in 1943. There is no evidence of the state-of-the-art in 1941. Perquin explains that RF agents were trained by British instructors, and also dependent on SOE equipment. “In Autumn 1941”, he writes, “following the numerous loss (sic) suffered by those specialists and considering how such losses were threatening the very existence of the networks, the SOE decided to create a security course in Grendon, Buckinghamshire.” Yet, if losses of agents were due to overlong transmission times, or failure to switch frequencies, one might think the problem could have been swiftly addressed through tighter discipline. Gubbins’s edict of winter 1943-44, after ‘it did not take him long’ to work out what was happening, simply seems absurd.

It appears that Foot and Perquin were using the same source, but it is not clear what it is. In Resistance, Foot declares his heavy reliance on Pierre Lorain’s Armement Clandestin (1972), a book that also appears in Perquin’s Bibliography, which was translated and published in English as Secret Warfare in 1983. Lorain gives a much more reasonable account of what happened, and it is worth quoting three paragraphs in full.

“German detection methods had made decisive progress in 2 years. In 1941 and 1942, the localization of a clandestine station was extremely difficult. It could be carried out only if the operator transmitted on the same days of the week, from the same site, and on the same frequency during several consecutive hours. Direction-finding operations were not yet automatic, and panoramic reception was non-existent. The scanning of all usable frequencies was necessarily very slow and left substantial gaps.

In addition, during the final approach, each Gestapo agent had to hide a heavy suitcase containing a receiver with a loop aerial under his coat. A Tirolean cap or Basque beret tilting down over his ear just barely hid an earphone. Their general posture aroused the curiosity of even the most naïve of passersby.

The arrest of a radio operator thus required long months of continual surveillance, the operation was complicated by the fact that if a clandestine operator was spotted in the unoccupied zone of France (controlled by Vichy), the Germans could only signal the suspect frequency to the French radio control group at Hauterive near Vichy. The latter promised to look into the matter, but secretly warned the clandestine station to move as quickly as possible, and then supplied the Germans with an almost completely false position.”

The Funkabwehr article I referred to before contains nothing about operations in France against SOE. I have been advised that the unit’s records reside somewhere in Moscow, so one cannot judge how much of Lorain’s account is true. Yet it seems as if Foot’s official history tries to deflect attention away from other systemic problems in SOE’s deployment of wireless. (His comments above need to be transferred en bloc to the state of the game in 1943 onwards, a period I shall cover in a later article.) A careful reading of Mackenzie would suggest that a number of severe problems affected both the F and R/F operations in France until 1942: a lack of radio expertise for establishing reliable wavelengths and schedules, leading to failed use; struggles with transporting and concealing the heavy equipment; inappropriate choices of agents who had unsuitable personalities; careless practices by the wireless operators, who were not always trained properly; inappropriate centralisation of transmissions because of shortage of equipment, leading to intense and long broadcasts; betrayal by agents (such as the notorious VICTOIRE); the unreliability of the local police in Vichy France. It was easier for SOE to blame German direction-finding.

And it seems more probable that other territories – and another enemy – were the arena in which the Reichssicherheitshauptamt improved its detection capabilities. As I shall explore, the Funkabwehr was provoked into quick reaction after Barbarossa (June 1941), as the Red Orchestra started tuning up, primarily in Northern France and Belgium. Colonel Buckmaster, who headed F Section, reported that, as late as August 1942, in the Occupied Zone, he had only two wireless sets, of which one was operational, while in the Unoccupied Zone, the numbers were six and four. In Belgium, however, the following distressing tale emerges, as German counter-action took place. In the First Quarter of 1941, two out of 9 sets had been captured and operated by the Germans: the figures for the next three quarters were 5 out of 6; 8 out of 8; and 7 out of 8. I shall return to the topic of whether German RDF advanced faster in Germany, because of the activation of the Red Orchestra after Barbarossa, and explore how soon operations in France were able to take advantage of such breakthroughs. Overall, my conclusion would be that the sluggishness with which SOE mobilised its wireless communications, and the slow but steady steps by which the Funkabwehr moved into action against Communist spies in the latter half of 1941, suggests that Foot’s suggestions of hyperactive German detection-finding in 1941 are premature, and that the losses were due to other causes.

In any case we know that SOE was inhibited by the fact that SIS controlled its cyphers and communications until June 1942. Up until then, it had had to accept whatever equipment SIS gave it – clumsy and heavy apparatus. As Foot writes: “Agents were not best pleased at SIS’s first offering, a plywood box that weighed some 45 lb. (20kg), already looked old-fashioned and contained a Mark XV two-valve transmitter fitted with a morse key, and its power-pack, a 6-volt car battery.” Foot does not describe the travails that agents lugging a 45-lb. suitcase around an unfamiliar terrain must have experienced, let alone the difficulties in setting up a suitable aerial without drawing attention to themselves.

The conclusion about SOE’s (and specifically Gubbins’s) track-record concerning wireless up to 1942 must be that the operation was needlessly clumsy. It cannot all be blamed on SIS.  I read A. R. B. Linderman’s Rediscovering Irregular Warfare: Colin Gubbins and the Origins of Special Operations Executive (2016) in the hope of acquiring some deeper insights. Linderman informs us that a Frederick Nicholls served under Gubbins as director of signals during World War II, but that is the only mention that Nicholls merits in the Index, and the story is disappointingly thin on wireless matters. Maybe the skills of Nicholls, who ‘had managed to establish wireless communications with the British Embassy in Kabul during the Third Anglo-Afghan War’ (which occurred between May and August 1919) were stretched by the exigencies of communications in Nazi-occupied Europe if that was his premier achievement. The clumsiness of SOE’s wireless strategy would however endure until the end of the war, as I shall explain in a later episode.

Major-General Sir Colin Gubbins

The Red Orchestra

While the Comintern and its allies had enjoyed successful experiences with illicit wireless transmission in the 1930s, Stalin’s purges of 1937 and 1938 had required much of the Soviet Union’s networks in the West to be rebuilt. It was not hard to find native Soviet sympathisers outside Germany, since the propaganda of communism as the only effective bulwark against fascism had worked effectively both on the disenchanted ‘toiling masses’ as well as on the guilt-ridden intellectuals. Since Hitler had either executed, incarcerated or forced into exile any members of the Party, or outspoken supporters of communist doctrine, Germany remained a more difficult country to penetrate. But neighbouring nations provided a rich source of potential spies and informants: many eastern Europeans found homes in the Low Countries and France, for instance, and were able to fade into the background without being conspicuous. Britain had its own nests of spies, of course, both from the older universities – who had successfully detached themselves from any association with the Communist Party of Great Britain – as well as more traditional working-class enthusiasts. But these eager adherents to the cause of the proletariat needed managing, and directing in their efforts. They needed intermediaries, and they need a mechanism for getting the fruits of their espionage back to Moscow.

Soviet espionage had three arms – the Comintern, the NKVD, and military intelligence, the GRU. David Dallin, in his epic Soviet Espionage (1955), informs us that, as early as late 1935, “Only a comparatively small Soviet apparat now remained in Germany: the greater part of the network had either been dissolved or moved abroad. The OMS had moved with the Comintern’s West European Bureau, the WED, to Copenhagen; the passport apparat had gone to the Saar, and Soviet military intelligence to Holland and France; the party leadership had migrated part to Prague and part to Paris.” Thus what survived the purges (with the GRU the most hard-hit) was still a very fragmented approach to intelligence-gathering, with no guarantee that it would be efficiently shared back in Moscow. In Volume 2 of his biography of Joseph Stalin, Waiting for Hitler, 1929-1941, Stephen Kotkin writes (p 496) that a dozen NKVD station chiefs abroad were arrested in 1937-1938, and that, in Berlin, ‘Stalin cleaned house, arresting nearly every NKVD operative there’. The GRU suffered even more, with 182 operational staff arrested in the same time-period. Yet the growing menace of Germany and Japan meant that, under Beria, a rapid repopulation of the networks had to be accomplished.

The International Brigades in Spain had constituted a useful source of potential operatives, as well as an opportunity to grant new identified to infiltrated agents, by virtue of the passports that had been stolen from Brigade members when they entered Spain. Alexander Foote was a famous example of such a footsoldier who was plucked from obscurity to be sent to Switzerland to received training in wireless operation from Ursula Kuczynski, agent SONIA. At the end of 1938, agents in their dozens started arriving in Europe, as well as the Far East and the United States. Like the Nazis, but with far more deliberation and craft, the Soviets chose, or allocated citizenship to, agents who would never arouse suspicion owing to domestic (Russian) nationality. The complex borderlands of the old Russian Empire provided a rich environment for muddled heritage and absence of reliable documentation, in order to allow unverifiable accounts of life-history to be passed off.

Accounts of training for wireless activity are thin on the ground. SONIA’s memoir (which in these technical aspects is probably much more reliable than in political observations, such as her absurd accusations of imperialistic infiltration helping to crumble the Soviet Union) is certainly not typical.  For she was respected enough to avoid the purges, and also had had a long experience in China as a wireless operator before being recalled to Moscow for leave and ‘discussions’ in late 1935. Her account is unfortunately very muddled in chronology, but it is educational in that it clearly identifies some of the problems that illegal wireless operators would experience anywhere in Europe. After a brief interlude with her family in London, she was then sent to Danzig, then a ‘Free City’, where she was instructed to ‘obtain residence permits, find work to legalise our existence, and set up our transmitter for radio contact with the Soviet Union’.

SONIA had been instructed how to build a transmitter in China, by her lover, Ernst, and claims that she received a response from Moscow immediately she set up her apparatus. Her task was to advise a group of labourers undertaking occasional sabotage at a shipyard building U-Boats in Danzig (where the Nazis were outrageously breaching the constitution that the city had been granted), and transmit on their behalf. At one stage, she and Rolf moved to a new house, but discovered that proximity to a power-station made signals inaudible, and she had to take her equipment to an apartment – a lesson that probably stood her in good stead later in England. Yet she immediately stumbled dangerously: the apartment block she chose was the residence of several Nazis, and one day the wife of them asked her whether the reception on her radio had been affected by interference. Her husband had told her he believed that someone was transmitting secretly, and was going to arrange for the block to be surrounded. SONIA even mentions triangulation of radio detection, which would have been a very early indication of the Nazis’ fears – and progress in allaying them.

Soviet ‘Sever’ Wireless Model

SONIA did not take the right steps, however. She broadcast again, from the same apartment at the same time, instead of the middle of the night when neighbouring radios would not have been on. She should have moved to a friend’s apartment, or returned to Warsaw. It appears that she was in awe of doing anything without Moscow’s approval: the outcome was that she was ordered to return to Poland as she could no longer transmit. Thus, when she met her boss, Comrade Andrey, in Warsaw, she asked to receive further training in wireless construction and use in Moscow. That need was reinforced by her receiving a severe electric shock one night, burning her hand. SONIA would pay two visits to Moscow during 1937 and 1938 (she admits that the details of each congealed into a blur). Her return to Poland was uneventful. She had to return to Danzig to help a comrade set up his transmitter, and admits that he was ‘slow on the uptake’, so maybe Moscow’s selection and approval processes for its agents were not very rigorous. Communist fervor may have been considered more important than intelligence and the right psychological profile. SONIA felt she was not accomplishing much: “The Danzig people had their own radio operator, the Bulgarian comrade produced little information. I only transmitted once a fortnight.”

In August 1938, it was decided to send her to Switzerland, where the plan was to infiltrate agents into Germany, to make contacts at the Dornier aeroplane factory in Friedrichshafen. And that is where the story of ‘Sonia’s Radio’ picks up, with her eventual successful establishment in Britain in the spring of 1941, and her activation as a wireless agent a few months later. She met up with Sándor Radó, who as agent DORA had been appointed head of the Swiss network, but had no wireless skills. In his memoir, Radó writes how Sonia visited in him in December 1939, and how the following month his radio contact with Moscow had been established. He also describes a visit in March 1940, set up by Moscow Central, by someone he knew only as KENT (see below). KENT spoke authoritatively about the necessity of secure wireless procedures, stressing the importance of changing the number and times of transmissions as often as possible ‘as the best protection against being located’. He added that operators should move around different residencies, as well. “Keep changing them if you can – but again, avoiding any kind of system. The thicker the fog, the better.” It suggests, again, that a prematurely intense fear of radio-detection capabilities existed with the Soviets, and that their listeners back in Moscow would be prepared to listen around-the-clock for their agents’ transmissions. But it was easier to preach such practices than to follow them.

The Soviet defector Walter Krivitsky also gave hints of subversive radio activity in Central Europe. In his memoir In Stalin’s Secret Service, he related how Marguerite Browder, the sister of the head of the US Communist Party, Earl Browder, had graduated from the school in Moscow that specialised in wireless competency, and had then been sent abroad as an illegal with an American passport issued in the name of Jean Montgomery. “During 1936-1937 she worked in Central Europe where she laid the ground for the establishment of our secret radio station,” he added, with an unhelpful lack of precision. If we can rely on Krivitsky, shortly before his recall to Moscow Sergei Spiegelglass, sent on a deathly mission by his OGPU boss Yezhov, tried to get Krivitsky to assist in the assassination of his friend and colleague Ignace Reiss. When Krivitsky demurred, he then asked Krivitsky to hand Browder over to him, as he had an ‘important job’ for her in France. The implication in Krivitsky’s rather fractured account is that he managed to warn Browder of what Spiegelglass had in mind for her, and that she was able to continue with her wireless activities.

In his biography of Kitty Harris, The Spy With Seventeen Names, Igor Damaskin informs us that the European network was issued with much more sophisticated wireless equipment at the end of 1936. Kitty Harris, who was Marguerite Browder’s sister-in-law, was brought back to Moscow for retraining in January 1937. She apparently showed little aptitude, and it was determined that ‘any more technical training would be a waste of time. She was later assigned to be Donald Maclean’s handler in London and Paris, where she specialised in photography.

Yet wireless usage in broader Europe at this time was sparse. It was not necessary. Moscow had its eye on the long term. The presence of Soviet legations or embassies in most capitals of the West provided a mechanism for information to be collected and then sent by diplomatic bag or courier back to Moscow. As a long-term measure, a wireless centre was set up in Brussels, where Trepper, as the new leader of the western organisation, replacing Walter Krivitsky, installed himself in March 1939. Yet, as Heinz Höhne tells us in Codeword Direktor, Trepper left it dormant, concentrating first on recruiting a team of informers, and enlarging his contacts with the world of business, the military and diplomacy. Even when war broke out, there was no quick change of operation. Only when Nazi Germany started its invasion of Belgium and the Netherlands in May 1940 did hasty adjustments have to be made. Even though the Soviet Union was in a non-aggression pact with Germany, its needs for information on Germany’s plans, and the reactions of France and Great Britain to Nazi movements, placed increasing pressure on Trepper and his cohorts to deliver.

Communication switched to radio sets when the Germans occupied Brussels, and the staff of the Soviet legation was withdrawn. In August, 1940, Trepper moved with his mistress to Paris, leaving there the unreliable playboy Sukolov-Gurevich, known as KENT, as the only agent capable of representing the GRU network. The Sokols were then recruited as wireless operators by the Soviet Embassy, and trained by someone called Duval. By June 1941, the Soviet Military Attaché, Susloparov, had moved to unoccupied France, and Trepper was in Vichy on the day that Germany attacked the Soviet Union. Meanwhile, in Berlin, more urgent plans were made in April 1941 to establish direct radio contact between the cells led by Arvid Harnack and Harro Schulze-Boysen, the Soviet spies in the heart of the Nazi administration. (Even if Stalin did not believe the rumours of a Nazi invasion, some of his intelligence officers were presumably more realistic.) In late May, two transmitters were sent by diplomatic bag from Moscow to Berlin, ‘one a small battery model and the other a large mains-powered set portable enough when broken down to fit in a suitcase’, as Costello and Tsarev describe. Harnack was chosen to be the operator, but declined, delegating it eventually to an engineer named Behrens, while Schulze-Boysen took up the challenge for his group, with much more eagerness, selecting a factory technician called Hans Coppi.

Costello and Tsarev report further: “The Berlin groups had established several safe locations on the upper floors of trustworthy colleagues’ houses in the countryside outside the city where the transmitters could be assembled and their aerials run up into the attics in order to communicate with Moscow. The Centre arranged to keep a listening watch on set hours and days of the month, which were multiples of the numbers four and seven.” Coppi received training from the local NKVD office, and successful transmissions were made in the beginning of June, and picked up and decrypted in Moscow. The infrastructure was in place when Operation Barbarossa was started. As Dallin records the situation: “This, then, was the setup on the eve of the Soviet-German war: a number of espionage agencies with radio facilities and sources of information, organized but dormant, in Belgium and Holland; rudimentary apparats in France and Denmark; a few trading firms established as covers in Brussels, Paris, and Geneva; a promising start in Switzerland; and a group of enthusiastic but inexpert operators in the German capital.”

Summary

Thus, as the wartime alliances solidified in the summer of 1941 (with the USA to join the Allies a few months later) mainland Europe entered its most intense couple of years of illicit wireless transmission and detection. Many agents – as well as dedicated wireless operators – did not have a suitable profile for the tasks at hand, and had been sketchily trained. The equipment they used was frequently clumsy and unreliable. The support structures behind them had not always analysed the variables of distance, sunspots, terrain, or mechanical interference in depth enough to define the wavelengths and times that they should best operate. They frequently disobeyed best practices in their transmission techniques, and ignored rules of basic spycraft. But they all probably had an exaggerated sense of the state-of-the-art of enemy detection and direction-finding techniques at the time, and how efficient it was, and certainly used such capabilities as an excuse for sloppy behaviour when agents were apprehended. All this would change very rapidly as the battle of wits intensified in the second half of 1941, when Nazi Germany honed its capabilities in the face of the Rote Kapelle activity. The major significant conclusion is that, as Germany intensified its capabilities for detecting the threat of domestic (or imperial) illicit wireless, Britain moderated its own home coverage. Through policy and organisational change, it concentrated much more on transmissions in mainland Europe, and on the interception and decipherment of official transmissions made by the Nazi war machine.

The final observation to be made is to note the anomalous attitude of British Intelligence towards its Nazi enemy during this period. While crediting an exaggerated efficiency and skill to the Abwehr’s counter-espionage activities, in the form of effective Radio Detection- and Location-Finding, it attributed the obvious ill-preparedness of the agents (training, language, identification papers, etc.) it sent to Britain to the stupidity and clumsiness of the same organisation. Yet, while priding itself on its superiority in both regards, the British intelligence services (in this case MI5, RSS & SOE) developed casual habits in its interception of domestic illicit wireless, and also sent agents to the continent who were likewise unready or unsuitable for the challenges of working in hostile territory.

(I am again grateful to Dr. Brian Austin for giving me guidance on matters of wireless technology. Any mistakes or misrepresentation are mine alone.)

Sources, and for further reading:

SOE in France by M. R. D. Foot

SOE, the Special Operations Executive by M. R. D. Foot

The Secret History of SOE by William Mackenzie

Resistance by M. R. D. Foot

Deceiving Hitler by Terry Crowdy

Soviet Espionage by David Dallin

Codeword Direktor by Heinz Höhne

Unternehmen Seelöwe by Monika Siedentopf

Rediscovering Irregular Warfare: Colin Gubbins and the Origins of Special Operations Executive by A. R. B. Linderman

Secret Warfare by Pierre Lorain

The Clandestine Radio Operators by Jean-Louis Perquin

Wireless for the Warrior, Volume 4 Clandestine Radio by Louis Melstee and Rudolf F. Staritz

The Third Reich is Listening by Christian Jennings

SNOW: The Double Life of a World War Spy by Nigel West & Madoc Roberts

Operation Blunderhead by David Gordon Kirby

Sonia’s Report by Ursula Hamburger

Codename Dora by Sándor Radó

The Duel by John Lukacs

Double-Cross by Ben Macintyre

Hitler’s Spies by David Kahn

Fighting to Lose by John Bryden

Deadly Illusions by John Costello and Oleg Tsarev

Secrets of the British Secret Service by E. H. Cookridge

Codebreakers: The Inside Story of Bletchley Park by Alan Stripp & Harry Hinsley

Bodyguard of Lies by Anthony Cave-Brown

Secret Days by Asa Briggs

The Searchers by Kenneth Macksey

The Spy With Seventeen Names by Igor Damaskin

In Stalin’s Secret Service by Walter Krivitsky

The Guy Liddell Diaries, edited by Nigel West

The National Archives at Kew, London

This month’s Commonplace entries can be found here.

1 Comment

Filed under Espionage/Intelligence, General History, Politics, Technology

Struggles at the Desktop

Monitoring the home security system at 3835 Members Club Boulevard

[Warning: This article may not be suitable for readers of a sensitive disposition. It describes encounters with information technology that may be disturbing to some.]

“Nowadays if there is an error in the input program the computer not only detects it but gives the approximate description and location of the error and recommends procedure for correction.” (Gerald S. Hawkins, in Stonehenge Decoded, 1965)

When IBM hired me as a trainee Systems Engineer in 1969, it was not because of my data processing skills. That day in late August, when I walked into the Katherine Street office in Croydon, Surrey (shortly before the branch moved into the new building on Cherry Orchard Road), I did not know the difference between a punched-card and a paper-clip. It was not a classical career beginning, and not a carefully-planned strategic move. In an indecisive third year at Oxford, I had applied to take the Certificate of Education after the completion of my degree in Modern Languages, but soon began to have doubts. On a weeklong visit to a local primary school in Purley, Surrey, before the first term started, I had innocently queried the headmaster as to why the classes did not appear to be learning multiplication tables by rote. “Oh, Mr Percy!”, he replied with a condescending smile. ‘We don’t do tables any more!” For this was the era of ‘child-centred’ learning, where every infant had to discover for him- or her-self that 7 x 8 resulted in 56, and so on. I recall the way that tables and mental arithmetic were drilled into my generation about fifteen years earlier, and how the pattern of number combinations has stayed with me ever since. In 1968, however, I was entering the world of Progressive Education.

Perhaps my aspirations were also checked by my term of teaching-practice. Having had a term of almost total inactivity, owing to my being on crutches because of a rugby injury, I was informed, in December 1968, that I was urgently needed as a replacement at Bognor Regis * Comprehensive School, as the previous teacher of Russian had been fired for getting one of his pupils pregnant. I did not learn the cause of the summons until I arrived: the school was also going through a painful merger of a grammar-school with a secondary modern, which also dampened what remained of my enthusiasm. Halfway through this term, I decided that a quick return to the classroom was perhaps not the most life-enhancing prospect to be contemplated. Taking advice from some outfit that suggested that my interest in chess, bridge, crosswords and logic puzzles might open up some doors in the computing industry, I secured interviews with some manufacturers, of which NCR and IBM were the most satisfactory. I took care to complete my Certificate of Education so that I could have a back-up career lest the corridors of business found my talents wanting.

[* Bognor Regis is a coastal town in West Sussex. It gained its regal addendum after King George V recuperated there, and the monarch’s dying words have been apocryphally reported as ‘Bugger Bognor!’. When Ursula Kuczynski (agent SONIA) needed a place for her children to stay while she returned to the Soviet Union in the summer of 1938, she left them with a friend in Felpham, which is part of Bognor. There is no truth to the rumour that I was in 1969 undertaking, under deep cover, some early sleuthing into Sonia’s contacts.]

Unfortunately, IBM was a little slow to snap up the opportunity to make me an offer, so I had to write to them to explain to them that this entrepreneurial youth was thirsting to make his contribution to the computing revolution. Perhaps the company was waiting for such a show of initiative, since I was rewarded with an appointment at the Head Office in Chiswick, to meet one of their Personnel Managers (no ‘Human Resources’ in those days: employees were certainly not ‘associates’, and customers were assuredly not ‘guests’). I was delighted to find that this benevolent soul had also studied Russian at Oxford. He started to quote me a quatrain of Pushkin’s, which I was happily able to complete. I passed the interview. I was in.

Before I started the eight-week basic training course at Sudbury, Middlesex, I had a week in the office, where I was directed to a small room, and given a Programmed Instruction text on IBM’s System/360 to work through. These matters were all rather daunting to me, and I recall I had to interrupt my study to ask the Systems Engineering Manager what the meaning of some concept was. It all comes back to me quite clearly: I wanted to know what was special about the sixteen ‘registers’ of any 360 computer system. Registers were (and no doubt still are) the mechanism by which the locations of computer memory were addressed, but they also seemed to have some properties that lent themselves to high-speed arithmetic. Somewhat confused, I asked the manager whether he could explain their nature to me. “Oh, I never really understood all that stuff”, he said. “I wouldn’t worry about it.” I think we adjourned to the squash court soon after that, and I gave him a good runaround in return.

The Systems Engineering class was tough. All new recruits were required to go through the same basic training, to make sure they were immersed into the IBM way of doing things. I recall a few students who had already served several years with IBM’s rival, ICL, and were thus already very familiar with the concepts and practice of data processing. Most of the graduates straight from university had scientific backgrounds, and had used computers in their laboratory work. There were times when I wondered whether I would make it. My ability to learn seemed to correlate exactly with the ability of the individual instructor to present topics in schemas that matched how my brain was able to integrate new ideas, namely very logically, with clear step-by-step evolution, and no grand jumps that left canyons of unexplored territory behind. Gradually, things began to make sense. I completed the three stages of the training about a year later, and was ready to roll.

Unfortunately, IBM was not sure at that time exactly what the role of systems engineers was, as anti-trust threats had meant that the company could not hand out systems engineering resources to its customers for free. At the same time, we were neophytes eager to learn by practical experience, while the projects we were given were haphazard, not always suitable, and not always educational. I soon learned that I liked coding, appreciated the value of well-designed and well-implemented systems, and became very frustrated with poorly written documentation. And I did have a knack for working out what was at fault when things went wrong, although that experience was marred by a disastrous project where I was asked to make some changes to a Vehicle Scheduling Package for a prominent and demanding customer. There was no guide to how the product worked, and I stumbled for weeks in trying to tweak it to meet the idiosyncratic needs of the customer. I received no help: the project was simply abandoned, I believe. But two lessons started to emerge in my mind: i) the knowledge that there was a logical explanation for every computer failure, and ii) the importance of good diagnostics being built into any product.

I move forward seven or eight years, and two jobs later. I was working as European Customer Service Manager for a small American software company. Our flagship product was known as a transaction-processing monitor, an adjunct to the operating system that handled communications with a network of terminals and managed the user programs that the customer wrote to provide on-line business functions. One of the challenges with this software configuration was that a motley set of technologies all operated in one partition, all clamoring for resources, and all potentially stepping on each other. Much of the code was written in low-level Assembler language, which provided greater manipulative power, and faster execution speeds, but also provided opportunities for corrupting storage occupied by other software. Frequent were the ‘core dumps’ (we still called them such, even though ferrite cores had been superseded as memory components by then) that were mailed in by customers when the system blew up, and we were unable to detect what had happened over the telephone. Then the support team would compare the state of computer memory with source listings of our product, in order to find out where our product (it was frequently the fault of the product) had gone wrong.

One particularly stubborn problem endures in my memory. A prestigious customer had experienced an execution failure, not recreatable, that caused the partition to explode. (The customer was actually the institution where the atom spy Klaus Fuchs, for whom Sonia had acted as courier in 1942-43, was working when he confessed in January 1950: there is no truth in the rumour that I was sent in by MI5, under subterfuge, to undertake an audit of its security procedures.) The requisite hundreds of sheets of print-out were sent in. No one could work out what had happened, and I devoted hours and hours to it. Eventually, I determined that it looked as if an error routine in IBM’s telecommunications package, VTAM, had failed to save properly the register contents that had been passed to it (and which had to be restored when the routine had completed its work), as all processes used those same registers I had been puzzled about back in 1969. I called the customer with my tentative suggestion, and asked him to pursue the matter with IBM. The next day he called back: indeed, one of the error handlers was incorrectly saving and restoring registers. He apologized for not searching for, and applying, the fix that would remedy the problem. Much goodwill was gained.

The second experience that reinforced my earlier lesson was in helping to roll out a new feature in the product, something called ‘Multiple Record Hold’ (MRH). The previous version had allowed only one file record to be held at a time, which was a heavy constraint. If a user application wanted to prevent anyone else accessing a customer record, say, while it then checked an inventory record that it might want to deplete, the systems designer was in a bind. MRH addressed that problem. But our developers designed and coded the feature too quickly and carelessly. Several occasions would arise where the programmer would try to invoke the feature inappropriately (for example, invalid keys, or multiple requests to the same file), or the software detected something illogical. It would return an ‘L’ code to the program, indicating such. But the programmer had no idea what it meant. There must have been several dozen places in the source code where an ‘L’ error code was returned. We, as support personnel, had to trace through the record of programmatic requests, and the source code listed, to detect at what point in the logic the ‘L’ had been returned, and then provide an explanation. But it could all have been made so simple: an auxiliary area existed where a return code could have been posted, and a corresponding piece of documentation could have explained what every code meant, with an enormous benefit in productivity. I was just about to start coding this enhancement when I was invited to work as Director of Technical Services for the parent company in Norwalk, Connecticut. At that time the flagship product was on the way out: the feature was never implemented. And so my wife and I, with ten-month-old son, moved to the USA.

[In parenthesis, for the more technical among my readership, I should also mention here that an unusual feature of this product was that the Control Program was written in a high-level language, COBOL, a decision presumably made in the interests of clarity and maintainability, not in the cause of performance. But when some advanced features were added to the product, it became necessary for the CP [not the Communist Party] to access low-level bitstrings, something COBOL cannot do. Thus an Assembler (low-level) language subroutine called GETBITS was added, to return statuses for further decision-taking and logic-branching. I recall very clearly how one of our most demanding – and shrewdest – customers in the UK, when undergoing performance problems, ascertained, through the use of a testing device, that GETBITS was consuming 6% of all machine cycles on its 370/145 – an enormous amount. Furthermore, when I inspected the new CP code, I discovered that, in many circumstances, the GETBITS routine was being invoked, but the CP was then taking branches that were completely independent of the results of the call! When I vaguely suggested to the President of the Company (who had probably written much of the original code himself) that I could rewrite the whole CP in Assembler language on my weekends, and deliver a much faster system to our customers, he declared, very seriously, that anyone who attempted that would be fired. He still relocated me to report to him in Connecticut, but later gracelessly told me that he only did so because the Director of R & D persuaded him. On such whims do whole lives change.]

The reason for this long introduction is that I recently had to replace my home PC, and experienced massive problems. For some months, my old HP Pavilion had been warning me of its imminent demise. The fan had broken, and the device was presumably in danger of overheating. I would get a warning message each time I re-booted, and occasionally Windows would blow up. So shortly before Christmas, I bought an HP Envy Desktop, preparing to install it after my winter break. I did not buy a printer or monitor: I had an HP Photosmart printer that was working well, and, only a year ago, I had had to replace the monitor that had suddenly died on me with a new model. This new monitor had HDMI support, but, since my PC was so old, it did not support an HDMI connection, and I thus had to use the older-generation VGA connector. This apparently meant that I had no sound support on my computer, but that was no great loss, even though I could not listen to music while I was working. I got used to it. Early in January, I thus loaded up the printer with new ink cartridges, backed up the files on the old PC, checked the cable configurations to ensure I knew what socket went in where, and unpacked the new machine.

To start with, all went very smoothly. True, Windows10 was a bit of a shock, with some features apparently dropped, and some weird patterns of activity occurring, such as random duplication of keyboard strokes. But overall it worked, and I restored my files (well, partially: but that’s another story.) Then I suddenly realised that I was not getting any sound from the computer, despite the new HDMI connection. The driver was okay, the system told me that the graphics was working properly, and yet no sound emanated from the monitor. It took me a while to work out that, all that long year ago, I had been sold a monitor with no sound support. Well, it was my fault for not asking, I suppose, but I think the salesperson was at fault, as well. Maybe he just wanted to move that product off the shelf. After all, why would I want to move from an antiquated broken monitor that supported sound to a spiffy new one that didn’t? There’s a lesson.

Next, I tried the printer. And here is where the problems started. Word documents would not print at all; PDFs would print, but very faintly. Crossword grids from the Web printed out partially. Emails from my queue printed fine, however. (I have one to prove it, as it relates to my problems.) What was going on with my device, which had been working so well a day beforehand? What caused such erratic behavior, where some items came out fine, but others were ignored? I did not believe it was a dirty printhead problem (something I had encountered and fixed a couple of years ago). My first step was, on my next trip into Wilmington (thirty-five miles away) to go to Best Buy, the store where I had bought both the printer and the PC, and ask for their advice. They immediately said ‘buy a new printer’, hinting that many users suffered from the same or similar experiences, as it would be too expensive to investigate the problem, and printers were so cheap. But I wasn’t going to give up that quickly.

After looking on the Web for users with similar complaints, I tried a number of things. I reloaded the printer driver (the current version was dated October 2015, which was perhaps not encouraging). I deleted the device, and added it back in. I reset it. I set it up as a default printer. I tried printing test pages. At some stage I logged on to the Microsoft and HP support forums, where ‘experts’ (but not employees of the respective companies) would generously offer suggestions to fix the problem. Nothing worked. Eventually, an HP employee joined the forum, and tried to help me. I shan’t go through all the steps he recommended, but he ended up giving me secret codes to enter on the printer itself, to determine why it wasn’t able to operate any off-line functions either. But even this process did not work as he outlined, as it was interrupted by another message. At this stage, we agreed that I should call up HP customer support.

Since the problem appeared to be with my newly warranted PC, I called the number for desktop computers, and was soon speaking to a support representative (in India), to whom I gave all the relevant information. Then, when I described my problem, he said that I needed to speak to the Ink-jet support group, and gave me another number to call. I went through the same process, was given a case-number, and started providing details of my problem. But when I gave the representative the Serial ID of my printer, she (in the US, this time) told me that I would have to pay for support, as the device was no longer under warranty. This did not completely surprise me – I have paid for such telephone support from HP beforehand – but I was not actually in the mood, given the trials I had already experienced, for having to pay for diagnosis that I really felt was HP’s responsibility. I somehow convinced her that she should at least provide an initial investigation of the problem for free. So we downloaded some software that allowed her to control my computer while I watched.

What happened next was rather disturbing. The representative asked me what make of router I was using, and when I responded ‘Ubee’, she expressed a degree of shock, almost one of recognition, as if the Ubee-Photosmart combination was a known toxic one. I tried to determine whether that was the case, but received no reply, as she started manipulating the Ubee tables on my PC. Clearly, she knows what she is doing, I said to myself. And then the connections were lost. First, the phone contact disappeared. She sent me a message indicating such, so I quickly sent her a text, imploring her to call me back. Then that connection went dead, too, and I was left stranded, with the shape of my router tables unknown, and the problem unresolved.

At least I had a case number. I called back, but this time was routed to another call-centre in India. Even though I gave the representative there the case-number, and told him what had happened, he claimed he could do nothing for me. I rung off in exasperation, hoping that the contact in the USA would call me back. But nothing happened. I suspect that the supervisor of the representative trying to help me in the USA had interrupted the process, probably reprimanding the young lady for not charging me for such support time, and thus had broken off all contact. I shall never know. Even when an HP customer relations person (who had presumably kept an eye on the forum, and had been alerted by the HP technician who joined it) contacted me afterwards, he was powerless to find out what had been going on. But to abandon a customer half-way through a process when the device was under the control of a remote technician was scandalous, in my unhumble opinion.

So I gave up, and bought a new printer, from Epson. Never again any HP products for me.

Perhaps it was all a strange coincidence, but one afterthought came to me. If my printer had enough intelligence in it that, when I ran out of ink, and inserted new cartridges, it could send a message in real-time to HP Central to encourage me to buy a replacement set, maybe it was also smart enough to detect that it was now being driven by a more modern, faster computer, and that a process akin to what we systems engineers used to call ‘graceful degradation’ should occur, so that the user would have to buy a new printer? That was the immediate recommendation of the technician at the company who sold me the printer, remember. After all, Apple has admitted slowing down its devices to preserve battery power, and Volkswagen fudged emissions when engines detected that they were running under laboratory tests. I would not be at all surprised if something like that happened.

And then my wife’s laptop computer started having problems. She would be told that an important Security update needed to be installed on Windows10, after which the process would hog her computer for hours on end, only to fail with the message ‘0x800700c1‘, when it was 99% complete. We ignored it for a while, since I was mightily consumed with sorting out my own PC, but I at last got round to investigating. ‘Contact Microsoft Support’ was the guidance, so I went on-line, and was soon directed to a document titled “You receive the error message ‘Something went wrong’, when attempting to install the latest version of Windows10.” I was amazed to learn that the company offered ‘many steps that I could try’, as there were ‘many possible reasons your device may be unable to update to the latest version of Windows’. This was extraordinary. A specific error message had been issued, yet the software had no clue as to what circumstances had cause it to fail, and the user of a consumer product was supposed to experiment with all these approaches in order to resolve the problem? What on earth would the Little Old Lady from Dubuque do?

I decided to request an on-line chat with a support person. This did not take long, and I was put in touch with Parthiban, in India. We set up the protocol by which such persons take over control of the computer, and he soon decided that the problem was due to a corrupt database, and a conflict with Norton Security. He initiated the update again, but he had to sign off before the process completed, leaving me with a link that I could invoke in case of failure. I was given a case-number, and waited for an hour or so. And then the installation failed again. So the next day, I used the reinvocation, and was before long involved in another on-line chat, with Deepthi. Now Deepthi did not appear to know what he (or she) was doing, as I could watch him wandering aimlessly around HP configuration options. My mistrust was justified, as he suddenly signed off the session without letting me know why.

Accordingly, the next day, I reinvoked the link, and noticed that I was 93rd in line, so decided to try again later. The queue had then diminished to 21, so I tried it again, and was soon engaged in an on-line exchange with Praveen. His diagnosis was that some cookies needed to be removed, and Norton Security had to be disabled for a while, as it was inhibiting the execution of the Microsoft Update routines. So I watched as he cheerfully went through the whole process leading up to the installation of the updates. Then he left me to watch for an hour, until the update failed again.

Yet, when I tried to re-invoke the link to resume my interchange, I was told that it was no longer valid. This time, I resolved to speak to a real person, called the support number, and, after a wait of about fifteen minutes, I described my problem to the support representative. She took my number, and soon I was talking to another agent, named Tony. (By asking him what time it was where he was working, I determined that he must be located somewhere in the Mid-West.) Anyway, while he seemed to be unable to look up my Call Number, and discover what approaches had already been applied, Tony sounded much more confident, and judged that I needed a larger partition size to run the routines. So I watched as he downloaded the Minitool Partition Wizard (how come Microsoft does not supply this facility?), which ran for about half an hour. That task having been successfully completed, he said he was going to re-install the whole of Windows10, so that I would not have to deal with a separate Security Update. I was getting a bit anxious as this process started, so I begged him to stay on-line until it completed, indicating that he could multitask with other customers while the update continued. Yet he was so confident that his solution would work, he said we should ring off: he did however commit to calling me in another hour to check how things were going.

Predictably, the update failed. After about an hour and a half of installation, verification, preparation and execution, I received a short message, with no diagnostic code: ‘Windows installation has failed’. And this saga would not be complete unless I informed you that, no, Agent 4 (Tony) never called me back, despite his promise. I had been abandoned again.

Before finally agreeing to give up completely, and simply to ignore the messages emanating from Microsoft that were constantly bugging my wife, as she worked at her computer, informing her that her security was at risk, and that updates still needed to be installed, I decide to post a plaintive appeal on the Microsoft Support Forum. I summarized all that had occurred, and expressed my frustration at Microsoft’s shoddy installation software, and its even more unprofessional support agents, who appeared to apply guesswork in trying to resolve problems, and repeatedly left consumers like me hanging dry. My appeal was quickly picked up by a Microsoft employee who has been very patient in going through my experiences. Yet his final recommendation, after I gave him the status of my Windows10 System Build, and maintenance applied, sounded very much like the process that Agent 4 had undertaken. When I pointed this out, he urged me to try what was (he said) a very simple process: indeed, he himself had written the on-line document that guided it. So I sat down, went through his steps, disabling Norton Security and trying again when that package told me that one of Microsoft’s modules was unsafe, and had had to be removed. About ninety minutes later, the Microsoft software, having gone through download, installation, verification, and preparation, started its execution. After half an hour, I received exactly the same message that had appeared in the previous try: ‘Windows installation has failed’.

The Forum Observer responded promptly, requesting that I send him (via OneDrive) a couple of log files from an obscure Windows folder. I am not sure why no one had thought of inspecting such data before (I had in fact suggested such a course of action several days earlier, as I suspected such files should exist somewhere). I had not used OneDrive (Microsoft’s file-sharing service on the Cloud) before, but I retrieved the logs, followed the instructions from my iPad, created the OneDrive link, and posted it on the Forum page.

And then I received the following amazing message from the moderator:

“A Windows upgrade requires DISM utility to work and in your case DISM fails which then triggers a rollback.

Error initializing DISM Session: [0x800700c1], [gle=0 x800700c1]

Right-click Start>Command Prompt (admin) and type in:

DISM /ONLINE/ CLEANUP-IMAGE/ SCANHEALTH

If that fails with 193 post back the DISM log present at C:\Windows\Logs\DISM\ again through Onedrive”

As John McEnroe would say: ‘You cannot be serious!” And don’t you just hate it when your DISM fails? So I went ahead, and yes, the SCANHEALTH failed with a 193, and I posted on the forum the link to the DISM log on OneDrive. Isn’t this exciting?

The next news was not good. My contact thought that the damage ‘was beyond repair, and that I would either have to reset Windows or do a clean install. He pointed me to another link, where a Mr Carmack had published a document titled ‘Clean Install Windows 10”. Mr Carmack attempted to sell the process by describing it as ‘a game-changing learning experience that will make you permanently the master of my PC’, going on to write that ‘to stretch this out over days or weeks you’ll learn better how each change affects performance.’ But typical home users of PCs do not have ambitions of becoming geeks, taking up Windows maintenance as a hobby. The only game I wanted changed was the one of getting Microsoft to fix its software. The steps that Mr Carmack outlined are monstrous (see https://answers.microsoft.com/en-us/windows/forum/windows_10-windows_install/clean-install-windows-10/1c426bdf-79b1-4d42-be93-17378d93e587), and must be very prone to error. And, even if I went through all this, what were the chances the problem would recur? I replied in this vein, thanking the moderator again, suggesting it was perhaps time to give up. ‘So what are the implications of simply ignoring the attempts by Windows to install the Security updates? Maybe the laptop should simply be replaced?’, I asked.

There is an easier way, replied the moderator. He outlined some other steps, recommending that I do a reset, ‘as it might remove the corrupt driver which is preventing the upgrade’.  He had no idea what might have caused the problem, and suggested yet another site ‘where the experts might be able to help you better’. But, if a driver has been identified as defective, I wondered, why could it not be replaced? At this stage, I concluded that I had had enough. My wife and I would live with whatever nonsense Microsoft imposed on us, and replace the laptop with something from Apple when the time came.

It was difficult for me to imagine that my wife’s PC was the only one on the planet undergoing such experiences. She is a woman in a million, I know, but I do not understand how her rarity should extend to the tribulations on her laptop computer. And the exercise also reminded me how little way the software industry (or Microsoft, at any rate) has come in fifty years. The company delivers an upgrade to a system that is in many ways incompatible with the previous versions, and it has disabled certain functions. The on-line documentation frequently does not match how the screens of system information appear, so one is left groping. The diagnostic codes given when the software encounters problems are meaningless and obscure. One can find jokey tutorials on YouTube, but they are badly designed, often delivered in mumbles, and do not explain enough about the Whys of a particular feature. The support personnel who try to help the bewildered consumer are poorly trained, not provided with proper tools, and thus engage in guesswork. And, of course, we fogies have to deal with tracking down those tiny labels with product serial numbers, pasted in the most inaccessible places on the equipment, that have to be read with a magnifying-glass.

What galls me even more is that we (in the USA, anyway) are currently facing a bombardment of in-your-face advertising from Microsoft that promotes its new expertise in Artificial Intelligence as ‘Empowering Imagination’. It depresses me to think how such technology will be abused by a company so obviously inept at managing the release and maintenance of its own software. Perhaps the techniques of neural networks should be applied to Microsoft’s own configuration and diagnostic problems before they are imposed upon an unsuspecting world? Yet again, we have been here beforehand. I recall the surge of enthusiasm about AI about thirty years ago, when all number of hyperbolic claims were made about the advent of rule-based systems. Now we hear it again, with all sort of nonsense about systems that will be able to teach themselves how to be more effective, and thus achieve all manner of breakthroughs in medical diagnosis, or fraud detection, or whatever. Computers can be programmed to give results that appear to reflect intelligence, such as beating grandmasters at chess, but that does not mean that they are inherently intelligent.

Maybe this generation of AI is different, but a caveat remains. A key principle of computing science has been the verifiability of systems – the fact that code must be inspected to determine whether the logic has been implemented according to specifications. (If proper specifications actually exist, of course, which is a whole other problem: see Multiple Record Hold.) Thus I used to experience the process of ‘structured walk-throughs’, where one’s peers would wade laboriously through the code a colleague had written to apply more stringent tests that might escape the test data environment. If the onus of decision-making has now been delegated to the computing system itself, who now takes responsibility when something goes wrong? I was both amused and perturbed to read, in the New York Times, earlier this month, how engineers at Google have started analyzing how computers using neural networks reach the conclusions they do, as if the experts are concerned about the level of auditability that these systems provide. “Understanding how these systems work will become more important as they make decisions, like who gets a job and how a self-driving car responds to emergencies”, the article declared. (I write this the day after the Uber self-driving car in Tempe killed a pedestrian during a test-run.) Their concerns are appropriate: I smell litigation over unexplained, and inexplicable, disasters. The paradox is that, if the processes of AI are verifiable, the technology is considered mundane and unimaginative, while, if they are not, it is uncontrollable and dangerous.  What do you think, HAL?

*          *          *          *          *          *          *          *          *          *          *          *

A few years ago, the Times of London informed me it could no longer issue a cheque for the occasional fees for published Listener crossword puzzles without my submitting a complex form that confirmed that I was a proper US-resident tax-payer. The cost to complete the forms required was almost as much as the crossword fee, so I didn’t bother. Last year, my bank in the UK (with whom I have had an account since 1965) told me that I would have to change my deposit account into a long-term instrument that would mature in three years, as it was no longer allowed to pay interest on accounts to overseas customers. This month, I received a letter from Barclaycard (with whom I have had a sterling credit card for about forty years) advising me that my account would have to be closed in early April unless I could provide proof of a residential address in the United Kingdom. Thus another convenience (for paying magazine subscriptions, downloading files from the National Archives, purchasing gifts, even ordering a copy of my own book from amazon to send to a reviewer – all in sterling) disappears. I have maintained my UK citizenship, have paid all tax at source, as appropriate, and have always declared all my (puny) UK-based income to the US Internal Revenue Service. It is comforting to know that the British authorities are cracking down on the real risks to currency and tax fraud, and thus discouraging me from any further investments or expenditure in the UK, while allowing all that other soiled money from Russia and other places to be brought into London for the purposes of acquiring valuable assets and helping the economy.

This month’s Commonplace entries can be found here.

1 Comment

Filed under Personal, Technology

Web Woes

Last January, I received an email purporting to come from my bank. It looked legitimate: it had a clean logo, in the right colours, but it contained a predictable spelling mistake, and did not originate from a valid bank email address. Yet I was concerned how the sender had obtained the fact that I was a customer of the bank, and gained possession of my email address. Had there been a serious security breach? Having occasionally received spoof emails from other institutions, which I forwarded to the address they gave for reporting such, and subsequently received grateful acknowledgments, I did the same with this one. I looked up the address to which such suspected spoofs should be sent (abuse@  . .  .) , and waited for a response.

And waited. And waited. I lingered a couple of days, and then sent another message to that address, inquiring whether the mailbox was being monitored, and requesting a reply. There was still no response, or even an acknowledgment. That was depressing, and utterly unsatisfactory. I thus went to the website again, trying to find a manager responsible for email fraud. The website was singularly unhelpful: it did not allow any chatroom discussion of security topics, and I entered a hopeless loop of going back to being invited to send further emails to the given ‘abuse’ email address. The bank provided no lists of executives to contact, no bank head office address to write to, only a couple of telephone numbers, neither of which looked suitable for my problem.

I tried one of the numbers, and after going through security checks, I spoke to someone (in Ohio or Iowa, I believe). She could not help me, but agreed to forward me to someone who could. I was thus transferred to a number in Atlanta, where I again introduced myself and my problem, and went through security checks. That person also decided that he was not in the office that could help me, but knew which section was responsible, and transferred me to another number.

I waited about twenty minutes before someone accepted my call. I again described my problem, and went through the same security checks. I was then told that that office was responsible for ATM security, but not for possible spoofing breaches. When I described my frustration to her, she said that she did not know what the policy was, but it was maybe unrealistic of me to expect any response from the Abuse department. I replied that these days it was very easy to set up an automated email reply system that would at least confirm that a customer’s message had been received, and indicate what kind of action was being taken, and added that it seemed to me that the Bank did not look as if it took reports of spoofing attacks, and possible security breaches, very seriously. She assured me that that was not so, and agreed to track down the Abuse Department. I was then left hanging on the telephone for another five minutes.

When she returned, she gave me the name and address of a ‘Resolutions Services and Support’ office, but no telephone number, no name of an executive responsible, and could not explain why that was not so. When I asked her what I should do next if I sent a letter to that office, and received no reply, she encouraged me to write ‘Response Required’, to ensure that I did receive a reply. This I did. But I was not hopeful.

Fifteen years ago, when the Web started to become a useful communications mechanism, corporate websites were full of data about organisation, functions, executives, addresses, telephone numbers, etc. Nowadays, it seems that their prime purpose is to provide a blatant marketing presence, and to make it extremely difficult for the inquiring customer (or prospective customer) to identify a department or person he or she might wish to contact. In addition, we have the blitz of customised advertisements: I cannot bring up the BBC website to check the cricket scores, or surf to a news site to ascertain Kim Kardashian’s views on this year’s Man Booker Prize nominations, without waiting for half a minute while dopey high-resolution advertisements for car dealerships half an hour away, that I am never going to visit, are loaded. Somebody, somewhere, is paying for all this, and will one day work out that it is all a waste.

After composing a letter, and sending it to the address given, I had one last try at finding a real person’s telephone number. Eventually I found one, in the Public Relations department. I called it, and left a message describing my problem (it was a Saturday), thinking I had done all I could. And then, out of the blue, a couple of hours later, I received a very polite telephone call from a Bank employee, who said that he was the Executive in charge of Security. His friend in the PR department had picked up my message, and alerted him to it.

As we discussed my problem, Mr. Watkins (not his real name) apologised, but said that, owing to the vast amount of spear-phishing emails that the Bank received these days, it had decided not to acknowledge any messages received from its customers, as it only encouraged more traffic that could overwhelm the system, and he started to brief me on the security challenges that any bank of its size has to counter in 2017. I responded that that might be so, but in that case why did the Bank simply not include some text to indicate that it inspected every genuine message that came through to its hotline, but that it would probably not respond individually to every item? Would that not provide for a better management of customer expectations?

At this stage, Mr. Watkins started to give me another little lesson about technology, at which point I decided to explain my credentials. While I am no longer au fait with all the issues to do with website maintenance and data security, I was one of the two executives who launched the Gartner Group’s Security product back in 1999. When I described my background, Mr. Watkins became even more amenable, and we moved on to a new plane. He seemed very proud of the fact that the Bank spends millions and millions of dollars each year on security. He essentially agreed with my recommendations, gave me his telephone number, and encouraged me to stay in touch while he investigated the problem.

Over the next few weeks, Mr Watkins was jauntily positive. There had been meetings, attended by database administrators, web designers, lawyers, security experts, public relations people – even manicurists, for all I know. It was important that everyone had buy-in to this significant portal of the bank’s business, and every detail had to be examined. And then, early in March, he proudly told me that the new functions had been implemented.

But they hadn’t. There are two entries to the bank system – a public one, and a subsequent secure sign-on that leads to a private area where customers can maintain their accounts. The Bank had attempted to fix the public ‘help’ area, where they had incorporated the text I suggested (although they made an egregious spelling mistake in doing so, spelling ‘fraudulently’ as ‘frauduleny’), but they had not touched the private zone. When I pointed this out to Mr Watkins, he was incredulous, and eventually I had to send him screenshots to prove that those spaces existed. I gently pointed out to him that it was as if the Bank’s executives had never tried to log on to their system as retail customers. He was suitably chastened, and promised to get back to me. More meetings with lawyers and psychotherapists, no doubt.

Nothing happened for a while. I continued to perform my on-line banking, and regularly checked the ‘Help’ section of the secure banking site to see whether it had been fixed. On March 20, Mr Watkins wrote to me as follows: “I’m writing as a brief status update to let you know that the changes you’ve identified below are scheduled to be implemented within the next 2 – 3 weeks.  In addition, I’ve had our team perform a comprehensive review of all of our web pages to ensure as much consistency as possible.  I will update you again once the necessary changes are complete.”

I waited again. No update from Mr Watkins, so six weeks later, on May 2, I emailed him again, pointing out that the unqualified advice still sat there, unimproved, in the private area, but did confirm that the rubric in what was called the Security Center was now clean and (reasonably) correct. (It had new spelling problems: ‘out’ for ‘our’, but no matter  . . .) I gave him the url of the offending area. Because of some personal issues, he had to hand my message over to his personal assistant to work on. He was under the impression he had already informed me about the changes the Bank had made.

I had to start again with Christine (not her real name). After she sent me an email informing me that the changes had been made, and how I should report suspicious emails, I had to explain to her that there was a discrepancy between the two zones, and I informed her of the fresh spelling problem. “Thank you for the feedback,” she replied. “We are currently working with our teams to review and will keep you posted.” More teams, more confusion. Less chance of a correct fix. I remembered Charles Wang of Computer Associates, who said once that, when a programming project started to drag, he would take a person off the team, so that it would run faster.

Another few weeks passed by. On May 25, I emailed Christine, and copied in Mr. Watkins, asking where things stood, only to receive the following reply from Mr Watkins. “I’ve tasked the multiple teams involved in producing and delivering these web pages to pull together a broad effort to reconcile all content.  These teams are currently researching what this will involve and we plan to meet back with them to discuss their assessments during the week of June 12. Please rest assured that there are no idle hands involved in this work but given the significant size and complexity of this effort, I’m focused on a) updating any current pages while b) ensuring the proper controls are in place to ensure ongoing alignment and consistency.”

Well, ‘resting’ I probably was, but ‘assured’ did not exactly describe my composure. I waited again. And then, on June 21, I learned from Christine that a new executive had been brought in to ‘address the issue going forward’ (as opposed to ‘going backward’, I suppose). I was invited to join a conference call, so that my concerns could be addressed. I declined, however. I did not need a conference call, and I instead carefully pointed out again that, while the problem had been fixed in the Privacy and Security Center, the text had not been incorporated in the private area, for which I provided the link again. All that Christine did was to provide me with instructions on how I should use the Bank’s web-page to report problems (as if it were not supposed to be self-explanatory by now).

I took one final stab at explaining the problem, pointing out how badly designed the whole website was, with its circular paths and inconsistent terminology, and I provided an explicit analysis of the problems with the Bank’s customer interface. I expressed my amazement that Bank officers could not identify the anomalies in the system, and fix them. I copied the message to Mr. Watkins.

On July 1, a new communicant appeared – probably not the executive brought in by Mr Watkins, as he introduced himself as being ‘on the team that oversees the on-line banking platform’. Arthur (again, not his real name) kindly provided me with a long explanation of all the changes that the Bank was introducing, including not just my recommendations, but many other improvements, as well. I thanked him, and promised to keep my eye open.

Well, it is now July 25, as I write, and the same old text appears under ‘Report Fraud’ in the private banking section, with no indication that messages will not be acknowledged. A simple change that I could have implemented on my own website in under five minutes (literally) still baffles the combined expertise of the Bank after seven months. Is this a record? Banks complain that they are stifled by regulation, but if they cannot even manage changes of this magnitude off their own bat, what hope is there for them? Is this story not an example of corporate incompetence and internal bureaucracy gone mad?

*                   *                  *                      *                      *                      *

The second incident concerns a recruitment at my old Oxford college, Christ Church (an institution, I hasten to add, for the benefit of my American readers, that is not actually the equivalent of Oral Roberts University, despite its name). The Hilary Term issue of the college magazine proudly announced that Christ Church was welcoming Sir Tim Berners-Lee as a Research Student and member of the Governing Body, with a mission to ‘grow Computer Science at Christ Church’. For those readers who might not know about Sir Tim’s remarkable achievements, I point you to https://en.wikipedia.org/wiki/Tim_Berners-Lee. He is known as the ‘inventor’ of the World Wide Web, and director of the World Wide Web Consortium, and took his degree at Queen’s College, Oxford. As a retired information technologist, I admire and applaud his achievements.

Yet some things that Sir Tim wrote in this promotional piece in Christ Church Matters puzzled and disturbed me. He characterised ‘several connected initiatives’ in which he has been involved for some time as Open Data, Open Standards, and Human Rights on Web. As an expert in data management for some decades (I was a data and database administrator in the 1970s, have experienced several generations of data-base management systems, was the lead analyst and product director for Strategic Data Management at the Gartner Group for a decade, and successfully forecast how the market would evolve), I believe I understand fairly well the issues regarding data security and data sharing. I found Sir Tim’s pronouncements about Open Data naïve and erroneous, and his thoughts on the role of Open Standards confusing, and maybe misplaced. But what really provoked me was what he wrote about Human Rights on the Web. “We have a duty to ensure that the Web serves humanity, and all of humanity”, he wrote, adding, somewhat rhetorically, about the concerns of the Foundation: “Is it [the Web] open, non-discriminatory, private and available to all, including minorities and women? Is it a propagating medium for truth and understanding, or more so for untruth and discord? Can these parameters be changed?”

Now I regard such questions as reasonably interesting, although I’m not sure what ‘minorities’ he was referring to (philatelists? Zoroastrians?), or why ‘women’ should come at the end of his list of concerns. But how could computer science be sensitive to such transitory social labels, or the gender of its users? Quite simply, what he proposes is either outside the realm of computer science, or lacking any toehold in what computer science has already generated about issues of data management (for instance, in the works of Sir Tim’s outstanding forbear, Edgar Codd, another Oxford man, an alumnus of Exeter College, and also a winner of the Turing Award). I found his pronouncements about serving humanity simply arrogant and pompous. Accordingly, early last March, I wrote a letter to the editor of Christ Church Matters, and to the Dean (whom I met last year, as my blog reported), which ran as follows:

“Am I the only reader of Christ Church Matters to be somewhat surprised, even alarmed, at the expressed rationale behind the new computer science initiative? The achievements of Sir Tim Berners-Lee are spectacular, and I have no doubt his intentions are honourable, but do the goals that he espouses not tread on the space of social advocacy, even corporate mission, rather than scientific investigation?

For example, the notions of ‘web-based data’, ‘Open Data’ and that ‘we [= who?] have a duty to ensure that the Web serves humanity, and all of humanity’ are certainly controversial. Data are not exclusively managed by web applications, but frequently shared. Indeed, it is a principle of good database design (a topic frequently overlooked in university computer science courses) that data be implemented for potential shared use, irrespective of delivery vehicle. There is thus no such entity as ‘Web-based data’. Professor Wooldridge’s statement that ‘when Governments generate data, there is huge potential value of that data is made freely available and open for all to use’ provokes enormous questions of privacy and security. To assume (as does Sir Tim) that ‘we’ can be confident enough to know how ‘all of humanity can be served’ has a dangerously utopian ring to it. Etc., etc.

The point is that technology is neutral: it can be used for good, or for ill, effect, and people will even disagree what those two outcomes mean. How is ‘all of humanity’ served when Islamic fanaticists can exploit web-based encrypted information-sharing applications to exchange plans for terror? Who benefits when private medical data is presumably made available for ‘all to use’? When is data private and when open? It is all very well for Sir Tim to assert that that his main motivation is ‘the personal empowerment of people and groups’ (is that phrase not both tautological and self-contradictory?), but that is a belief derived from his own sense of mission, not from a perspective of scientific inquiry.

Maybe these matters have already been discussed, and have been resolved. If so, I think it would be desirable to have them explained publicly. I believe those helping to fund such initiatives should be made aware that the boundary between science and evangelism appears to have shifted considerably.”

My letter was kindly acknowledged by the Dean, with a promise of follow-up, but I have heard nothing more. I suspect that I am seen as a minor irritant, getting in the way of some serious boosting of the college reputation, or maybe hindering access to vital government funding. But the question remains. There are researchers into computer science, and there are commercial enterprises. They frequently enjoy a symbiotic relationship, but there comes a time when enterprise have to make risks and decisions that go beyond what consortia and standards-groups can achieve. Ironically, Sir Tim’s statements about benefitting humanity sound uncannily like those of Mark Zuckerberg, the CEO of Facebook, who also has evangelical designs on improving the world. But the rest of us should be very wary of anybody who claims that omniscience to know how ‘humanity’ is best served, and who appears to be unaware of the Law of Unintended Consequences. And computer scientists should not start dabbling in evangelism.

*                      *                      *                      *                      *                      *

Regular readers of this website will recall my reference to The Trinity Six, by Charles Cumming, in my March blog. Since then, I have read his first Thomas Kell novel, A Foreign Country, and this month, the follow-up A Colder War (published in 2014), both of which I recommend. (Although I do not understand why we need to know every time Thomas Kell lights up a cigarette, or that he throws the butt of one into the Bosporus.) But my point here is to describe how unmistakably set in time these thrillers are – not so much by the political climate, although Iranian nuclear secrets and rebellious Turkish journalists give one a sense of that  ̶  but more by the use of technology. For the narrative is densely imbued with BlackBerries, iPhones, Facebook, TripAdvisor, SIM cards, SMS and O2 services  ̶  but not the dark Web, Snapchat or Twitter (or even Sir Tim’s Open Data initiative). Will it make the book soon seem dreadfully outdated, or will it be praised for its verisimilitude?

The pivot of the plot is indeed one such technological matter. (Spoiler Alert.) In what appeared to me as a very obvious mistake by the hero, an unencrypted text message leads to the eventual betrayal. And one other passage caught my eye  ̶  for a different reason. Cumming writes, about a surveillance operation at Harrod’s: “While most of the members of the team were using earpieces and concealed microphones, Amos had been given an antediluvian Nokia of the sort favored by grandparents and lonely widowers. Kell had banked on the phone giving plausible cover.”

I recognized that scene. Three or four years ago, I went into a branch of my bank to pay in a cheque (it may have been a check). The cheerful spirit behind the counter asked me whether I knew that I could pay in checks via my cell-phone (or mobile, as it would be known in the UK). Without saying a word, I then solemnly produced my venerated Motorola C155, manufactured ca. 2005, reliable, rugged, and not very handsome, and showed it to the woman. She then let out an enormous giggle, as if to draw the attention of her co-workers to this antediluvian instrument. As can be seen, it looks more like the shoebox phone from Get Smart (the 1960 TV series, not the 2008 movie).

But it did its job – just made and received phonecalls. My carrier forced me to replace it a couple of years ago, but, my fingers are too stubby for the keypad on the new thin model, and I never use my phone to access the Web. Enough woes in that. I miss my C155  ̶  ‘as favored by grandparents’.

*                            *                      *                      *                      *

Another saga started. In May, I had received a letter from History Today, inviting me to renew my subscription on-line. “Renewing your subscription couldn’t be easier”, it boasted.   I thus logged on to its website, but was frustrated in my attempts. I sent an email to the publisher, listing my failures. I explained that the system did not recognise that I was in the USA, did not allow me to enter my subscription reference, and quoted a sterling fee rather than the $99 mentioned in the letter. And when I signed on to my account, it gave me no option to renew, just to upgrade to access to the archive.  I received a prompt reply, which merely stated that the website had been going through some maintenance, but that once this were completed, I should be able to renew my subscription on-line.

I held off for a while, and then received another letter in the mail, which again proclaimed that ‘renewing your subscription couldn’t be easier’. It offered a price of $79, which I interpreted as a special offer, maybe making amends for the earlier technical problems. I thus logged on afresh, and made the renewal, but did notice that the confirmation came through with a charge against my US dollar credit card for £99. An obvious mistake, no doubt to be cleared up simply. I sent an email pointing out the error. After a couple of days, I had received no response apart from an email confirming my renewal, and encouraging me to contact the sender (the third name in as many messages) if I had any problems. I thus sent off another email, pointing out the discrepancy between the amount specified in the invitation letter, and somewhat impatiently requested a credit to be made against my credit card.

Yet another name replied, with the following message: “Thank you for your recent email.
I can confirm the reason they are different amounts and different currency is because it has been converted from USD to Pounds. So it will always show what we have received as payment here is England rather than the amount you paid is Dollars. If there is anything else that I can help you with please don’t hesitate to contact me.”

So, as the month wound down, I sent another message, pointing out that a fee of $79 would convert to £61, not £99. I am awaiting their reply. It is possible, I suppose, that they mistakenly took the exchange rate as 1.31 pounds to the dollar, rather than vice versa, although the letter lists the optimal online archive upgrade as a more accurate £30/$45. We shall see. If e-business speeds are predictable, I shall probably be able to provide an update to this transaction in January 2018.

The next episode of Sonia’s Radio will appear at the end of August. This month’s new Commonplace entries appear here.

Leave a Comment

Filed under Literature/Academia, Media, Personal, Technology

On Privacy and Publicity

While reading Robert Tombs’ superlative The English and Their History, I came across the following sentence, describing Samuel Johnson’s and Richard Addison’s London: “The mix of commerce and culture produced what has been termed ‘the public sphere’ – places and institutions for exchanging information and forming opinion, which lay between the purely private world and the official realm”. What could be more representative of that sphere in twenty-first century Britain than the pages of Prospect magazine, ‘the leading magazine of ideas’, as it promotes itself?

The February issue of Prospect included an article that outlined what has to be done with technology – primarily that concerning the use of social networking – to keep the citizens of the UK safe while protecting their liberties. The following earnest and superficially innocuous paragraph caught my eye: “The big technology companies have a crucial role – and unique responsibility – in building the security that keeps us free and safe. We trust them in part because they are private. Co-operation is much preferable to legislation. The next step is for all parties to collaborate on a way forward to benefit from new technologies while doing what we can to stop those who would do us harm. This kind of co-operation between public and private sectors is needs in free societies where security underpins our privacy, private enterprise and liberal democracy.”

But this simply will not do. To begin with, this contrast of ‘the public sector’ and ‘the private sector’ is hopelessly naïve. Whereas a government (or its civil servants) may be said to represent the populace, there is no such entity as ‘the private sector’ that may be negotiated with. A free market consists of a number of competing entities trying to differentiate themselves. Politicians frequently display a very wooden understanding of how markets work: I recall David Cameron’s meetings with ‘industry leaders’ to discover what it is they need from government. But what today’s leading businesses want will be protection in some way from any upstarts who threaten their turf. The needs of the market are not the same as the needs of current market-leaders. (Think of Norwegian Airlines threatening the established transatlantic carriers.) The FBI made the same mistake in thinking it could negotiate with ‘Silicon Valley leaders’ as a method of resolving this problem of encrypted information on PDAs and cellphones. This echoed the policy of President Obama, who in 2015 made a point of trying to ‘cooperate’ personally with Silicon Valley on these issues. Just this week, Obama officials again met representatives from technology and entertainment companies (but not chief executives) to discuss ways of combating extremists on-line. They still do not get it. This is a matter of law – to be addressed either by an interpretation of existing laws, or by new legislation. Parliament, not parleys.

For example, had a similar advance been suggested to computer technology leaders twenty-five years ago, the list of vendors would have probably included IBM, ICL, Data General, DEC, Wang, Honeywell, Siemens-Nixdorf  . . .  Apart from IBM, where are they now? Apple is presumably the IBM of today, but there is no guarantee that the ‘big technology companies of today’  (e.g. Facebook, Google, Snapchat, Twitter and Buzzfeed? – my computer industry advisory panel supplied me with these names) will dominate in ten years’ time. How long ago were Nokia and Blackberry the leaders in personal networking, for example? So how can such a suggested initiative encompass the coming vendors of tomorrow? Schumpeterian creative destruction is always at work.

What’s more, it would be illegal. Since most of the companies affected are American, any move by such to meet to discuss shared endeavours would have to be considered under anti-trust legislation (something that should probably have taken affect with Obama’s Affordable Healthcare Act, by the way.) For such companies to ‘collaborate’ with government to define pseudo-voluntary technology ‘standards’ (that would then be implemented at the whim of each company’s R & D design and implementation schedule) would be called for exactly what it is – conspiracy. And this aspect does not even touch the issue of whether such measures would be effective – which I shall not get into. This issue has been gaining intense attention in the past month, when Apple’s Tim Cook has again been assailed by the US Department of Justice. Cook has spoken out vigorously with the opinion that any back-door capabilities into a supplier’s encryption system would be abused by the bad guys. At the same time, Apple is planning for greater encryption of customers’ data in its ‘cloud’, which will make things even more difficult for law enforcement. (‘Ou sont les nuages d’antan?’) Yet in an Op-Ed piece in the New York Times on February 23, William J. Bratton and John J. Miller gave as their concluding argument for demanding that Apple should unlock its iPhone that Google and Apple ‘handle more than 90 percent of mobile communications worldwide’, and thus should be accountable for more than just sales. If such a rule does apply, it should apply to everyone.

So who is the supposed expert making this fanciful suggestion of bonhomous co-operation? Step forward, Sir John Sawers, ex-head of MI6, who indeed wrote the article. Not only that, Sawers advertises himself as having been ‘Chief of the Secret Intelligence Service (MI6) between 2009 and 2014’, and his second paragraph reminds us immediately of his credentials: “As MI6 Chief, my top priority was identifying terror attacks against Britain planned from abroad.” Sawers is then described as being the Chairman of Macro Advisory Partners.

What in heaven’s name is the ex-head of MI6 doing exploiting his past career while claiming to be an independent consultant? And how can he suggest that his role therefore gives him some credibility in representing the requirements and desires of the ‘public’ sector? There cannot be a more private organisation than MI6, whose very existence was withheld from the British public until 1994, of which no archival material has been released after 1949 (the year where the authorised history stops), and whence any retiring head a decade or two ago would have quietly folded his tent, picked up his ‘K’ (although Sawers had that already), and shimmied off to Torquay to tend his geraniums and take up square-dancing. Now such persons write their memoirs – surely in contravention of the Official Secrets Act  ̶  and pontificate with the chattering classes in the press.

This dual role of subtly promoting MI6 connections and policy, and claiming to be an independent advisor, does not sit well with me. Can MI5 and MI6 not speak openly themselves about such policy? What do they think of this grandstanding and self-promotion, I wonder? Or has Sawers undergone some shift in position now that he has left his official intelligence hutch behind? If so, shouldn’t he describe what that is?

It gets worse, in a way. A quick search on the Web for Macro Advisory Partners shows that the firm has a Global Advisory Board of seven (see http://www.macroadvisorypartners.com/the-firm/global-advisory-board ), of whom the prominent names are Kofi Annan (seventh Secretary-General of the United Nations), David Milliband (of Labour Party renown, and now President and CEO, International Rescue Committee), and William J. Burns (President of the Carnegie Endowment for International Peace, an institute which regrettably sounds like one of those Soviet fronts of the late 1940s: indeed, the Soviet spy Alger Hiss was its President between 1946 and 1949.) I didn’t see Cherie Blair’s name there yet, but she is no doubt a very busy woman. Sawers was Britain’s permanent representative to the United Nations between 2007 and 2009, so he no doubt developed some good contacts then. But is he running the show, or he taking his advice from this group of Kumbaya do-gooders? How will his undoubted steeltrap mind have been affected by such company? No wonder his recommendation for solving the technology problem is to get everyone around a table in peace talks.

I believe this is all highly irregular. Sawers surely has a pension that he can live off comfortably: he does not need this jump into the ‘private’ sector, where, ironically he can be much more expansive about his ideas than he was when working for the government. The undoubted impression that casual readers will gain from this promotional journalism is that there is some consistency in MI6 policy from the Sawers regime to the current set-up. That must make it very difficult for the present leaders of MI6 – and MI5, of course – to develop policy and work it through the normal processes, dealing with this distracting noise in the media. If they agree with what Sawers says, are they admitting that they are likewise influenced by pollyannaish internationalist wishful thinkers, instead of by steely pragmatism? And if they disagree with him, what does that say about continuity of purpose and perspective within MI6? It is all very messy, and, in the jargon of today ‘unhelpful’. Sawers should not have been allowed to exploit his past experience for monetary gain, and should have been prevented from entering the public sphere in this way: his employers should have insisted on a more stringent termination agreement.

Lastly, all this reinforces the unhealthiness of the transfer of careers between government and industry, and also demonstrates how absurd the UK Honours System is. ‘Captains of industry’, managing directors of private companies publically traded, should be looking after the interests of their shareholders. They do not provide ‘services to the industry’, for which gongs are awarded.  In addition, they have their own generous rewards, being almost without exception overcompensated by crony boards of directors, and remunerated handsomely even if they fail. Public ‘servants’ (who all too often act as if they were our masters) should be expected to perform their jobs well: if they do not, they should be fired. And when they retire from highly-important positions, they should do exactly that – retire.

*          *          *          *          *          *          *          *          *          *          *          *

Readers who followed my representation to the New York Times in my December blog may be interested to know of the follow-up. Having gained no satisfaction from the Public Editor (Margaret Sullivan), I wrote an email to the Executive Editor, and then one to the CEO, Mark Thompson. These attempts having resulted in not even an acknowledgment, I then sent a letter to Mr. Thompson, with a copy to the publisher, Mr. Sulzberger. Again, I have failed to extract even an acknowledgment from either gentleman. Did Mr. Thompson learn such manners at Merton College, I wonder?

I have since challenged the Public Editor on the Times’s somewhat irregular decision to give Madeleine Albright the opportunity to explain away her Clinton election campaign gaffe (about women supporting other women lest they go to hell) in an Op-Ed column. Again, no reply. And then, Ms. Sullivan announced earlier this week that she was leaving the position early to join the Washington Post. Am I entitled to imagine that perhaps she became frustrated in dealing with the bizarre journalistic principles at the Times, and that the paper’s failure to act on my complaint pushed her over the edge? (‘Dream on, buster.’ Ed.) As for Mr. Thompson, he left a mess behind at the BBC, and I expect further messes at the Times. This week, the paper ran a story about the post-mortem at the BBC over the matter of protected ‘stars’ like Jimmy Savile, who were allowed to get away with sexual malpractices in a corporate culture of fear at a time when Mr. Thompson was Director-General of the BBC (2004 to 2012). Mr. Thompson’s responsibility for that culture – or even the fact that he led the organisation –  was omitted from the article.

In conclusion, I highlight an item from this month’s Commonplace entries, taken from Hugh Trevor-Roper’s waspish Wartime Journals: “The Christ Church manner, that assumption of effortless superiority, is said to be galling to those who weren’t at Christ Church. But we can’t expect the world to be run for the benefit of those who weren’t at Christ Church.” Indeed.  Stop looking shifty, Thompson.                                            (February 29, 2016)

1 Comment

Filed under Espionage/Intelligence, Politics, Technology

The Myth of Buying Market Share

A few years after I became an analyst/consultant at the Gartner Group, I was introduced by one of the DBMS vendors to the thoughts of Geoffrey Moore, who had some original ideas about the challenges of high-tech companies in introducing their disruptive products to mainstream buyers. His book, ‘Crossing the Chasm’ (1991) quickly became a classic in technology circles (see https://en.wikipedia.org/wiki/Crossing_the_Chasm), and I adopted his ideas in evaluating and guiding the strategies of companies in my bailiwick. Some CEOs claimed to be familiar with the theories, and even to putting them into practice, but since the distinct message in the early years of the Technology Adoption Life Cycle was ‘focus’, they understandably struggled to keep their companies in line. ‘Chasm’ thinking requires a proper marketing perspective, but independent VPs of Marketing in technological start-ups are a bit of a luxury, and VPs of Sales always think of Marketing as something that supports their Sales Plan, rather than of their Sales Plan as something which realizes the Marketing Plan. Trying to close a deal to unqualified and unsuitable prospects is frequently an exciting challenge for such types.

As my career at Gartner wound down, and I considered retirement, I chose to move to a small software company in Connecticut. I was quickly brought down to earth: as a Gartner consultant, I had earlier been engaged by the company for a day’s work, at quite high fees, during which the managers attending dutifully wrote down all I said, and nodded appreciatively. When I became an employee, however, and started suggesting (as VP of Strategic Planning) to the CEO how she might want to change some of the processes (such as not having the R & D plan changed each month after the latest visit by a customer or prospect to the development facility in Florida), I was swiftly told: ‘You don’t understand how we do things around here, Tony’. That was not a good sign. So I picked up my thinking about Chasm Crossing, tried to talk my CEO out of an acquisition strategy (devised to show muscle to the Wall Street analysts, but in fact disastrous), and reflected on how financial analysts misled investors about markets. I had learned a lot from the first software CEO I worked for, back in the early 1980s, but he was another who didn’t understand the growth challenge. ‘Entrepreneurial Critical Mass’ was the term he had used to persuade his owners to invest in an acquisition strategy that was equally misguided: I had had to pick up the pieces and try to make it work.  (This gentleman was also responsible for bringing to the world the expression ‘active and passive integrity in and of itself’ to describe the first release of a new feature, which presumably meant that it worked perfectly so long as you didn’t try to use it.)   My renewed deliberations now resulted in an article, titled ‘The Myth of Buying Market Share’, which explained how completely bogus estimates of ‘market size’ misled CEOs and investors into thinking that all they had to do to be successful was to pick up a portion of a fast-growing ‘market’. I believe it was published somewhere, but I cannot recall where.

I reproduce the article here. I have not changed a word: it could benefit from some tightening up in a few places, and some fresher examples, but otherwise I would not change a thing, even though it is now sixteen years old. At the time I wrote it, I contacted Geoffrey Moore, and sent him the piece. We spoke on the phone: he was very complimentary about my ideas, and we arranged to meet for dinner in San Francisco, where I was shortly to be attending a conference. I vaguely thought that I might spend my last few years actually putting into practice some of the notions that had been most useful to me in my analyst role, and wanted to ask Moore about opportunities at the Chasm Group. So, after the day’s sessions were over, I approached him, introduced myself, and said how much I was looking forward to dinner. He was brusque – dinner was off. Obviously something better, somebody more useful, had come along. I was for a few minutes crestfallen, but then realized that I would never want to work for someone who behaved that rudely. I resigned from the software company a month later and began my retirement a bit earlier than planned. Since then I have never touched the industry again, apart from one day’s work for another small software company in New Jersey that desperately needed help, and wanted to hire me as VP of Marketing after I did a day’s consulting for them. North Carolina beckoned, and I have never regretted getting out when I did.

After receiving a fascinating observation from a reader (via Nigel Rees), I have posted an update to my piece on ‘The Enchantment’. The normal set of Commonplace items can be found here.                                                                                                                   (January 31, 2016)

1 Comment

Filed under Economics/Business, Personal, Technology

The Congenial Richard Dawkins

When I was in my early twenties, I read a book titled something like ‘Why Darwin Is Wrong’. It wasn’t a creationist text, but a popular science-based analysis. I can’t find the volume on abebooks (which doesn’t appear to list anything before 1981), but I recall quite clearly two of its major objections to Darwinian thinking, so far as the author understood it. One, that the notion of ‘The Survival of the Fittest’ (which was actually coined by Herbert Spencer to describe Darwin’s natural selection) was tautological, and thus meaningless, since what was ‘survival’ but another way of saying that an animal was ’fit’?  Two, that if the energies that contributed to survival took place after the animal had passed on its genetic material to its offspring, there would be no mechanism by which more adaptive traits would endure in the species.

I thought at the time that these points had merit, yet I was not completely discouraged from accepting that natural selection was the most plausible explanation for evolution, even though the exact mechanisms by which it occurred were still somewhat mysterious. I was, however, dismayed by another misconception, namely the way that the Theory of Evolution was frequently misrepresented as something purposeful by even the most knowledgeable of experts. I can recall the great David Attenborough, in Life on Earth, explaining certain phenomena in terms such as: “Thus, in order to survive, the bats had to develop radar.” This notion of purpose in Evolution is obviously nonsensical, and I have occasionally had to write to the Science Editor of the New York Times to point out where their journalists mistakenly ascribe this sense of an objective to adaptive changes. After all, did certain winged birds develop their flightlessness in order to make their life less hazardous? And what was the timescale according to which such adaptive changes worked? How long would it take for various initiatives to fail or succeed before the lack of ‘fitness’ wiped out the species? At the same time, as Jonathan Weiner’s The Beak of the Finch showed, describing the researches of Peter and Rosemary Grant on the Galapagos, small changes in the dimensions of finches’ beaks could rapidly take place in the light of changing climatic conditions and food supply.

Then Richard Dawkins’s Selfish Gene came along and changed everything, showing that the gene, not the individual organism (as Darwin believed) was the unit of natural selection. I have enjoyed Dawkins’s books since, although I found his first volume of autobiography, An Appetite for Wonder, rather scrappy and chippy. Now I have just finished his sequel, Brief Candle in the Dark. This is a new Dawkins. I think his PR firm must advised him not to be so offensive and controversial, because he positively oozes congeniality, and is nice about nearly everybody, and not nearly as scathing about religion as he used to be. (There must be a social meme in such superstitions that aids the survival of certain groups, a sad but unavoidable truth.) He also turns out to have almost as many friends as did Denis Healey or Lord Weidenfeld, and appears at times unbearably smug. As a curmudgeon myself, maybe I preferred the traditional Dawkins.

He has some fascinating new insights about the evolutionary process. I was interested to see what he had to say about the hot topic of epigenetics (defined in Chambers as the ‘gradual production and organisation of parts’, which is the study of how gene behavior is affected by environmental factors), and how he contrasted it with preformationist thinking (i.e. that, in essence, a homunculus was inside every human embryo). It seemed to me lately that some neo-Lamarckians, interested in promoting the notion of the passing on of acquired characteristics, have latched on to the term of ‘epigenetics’ to assist their cause. A footnote (p 402) from Dawkins is worth citing in full: “Don’t by the way be confused by the fact that the word ‘epigenetics’ has recently been hijacked as a label for a fashionable and over-hyped idea that changes in gene expression (which of course happen all the time during the course of normal embryonic development, otherwise all cells of the body would be the same) can be passed on to future generations. Such transgenerational effects may occasionally happen and it’s a quite interesting, if rather rare, phenomenon. But it’s a shame that, in the popular press, the word ‘epigenetics’ is becoming misused as though cross-generational transmission was a part of the very definition of epigenetics, rather than a rare and interesting anomaly.” Thank you, Professor. Just what I was looking for.

In one area however, I wonder whether Dawkins has got it wrong. I recall, at about the same time that I read the book on Darwin, taking in another work that pointed out how quickly scientists make analogies between the human body and whatever the current state of technology is (i.e. a pump in the 17th c., a clock in the 18th , an engine in the 19th , a computer in the 20th ). I thought that it might have been Arthur Koestler in The Ghost in the Machine, but I can find no trace of it there, and in those pre-spreadsheet days I did not keep track of every book I read. No matter: I think the point is valid. And Dawkins falls into the same easy motion. On page 382, when discussing the possible source of language, he makes the claim that ‘the human brain must possess something equivalent to recursive subroutines’ (an ability for a computer program to call itself and then return to an outer version of itself), a feature he says exists in Algol 60, but not the original IBM Fortran  language he used. Such a feature in human genes, which he calls ‘macro-mutation’ might have come about in a single mutation, and could have been responsible for the ability to create the phenomenon of language syntax. In reducing a complex organic process to a mechanical one, however, I believe Dawkins makes a categorical mistake. A computer program is only an artifact of the entity that he is describing, namely the human brain, which is a far more complex phenomenon than the strings of ones and zeroes that comprise a language compiler. His comparison is therefore merely crude reductionism.

But then Dawkins compounds his error. He goes on to write: “Computer languages either allow recursion or they don’t. There’s no such thing as half-recursion. It’s an all or nothing software trick. And once that trick has been implemented, hierarchically embedded syntax immediately becomes possible and capable of generating indefinitely extended sentences.”  First of all, if it is a design feature, it is not a trick. The trick – if there were one – would be an inherent flaw in the software where recursion did not work properly all the time – either by faulty implementation, or by a deliberate clandestine approach that made aberrant decisions based on some external circumstance or internal control data. After all, we each one of us know now about the Volkswagen Emissions Control Software, which gave false readings when the engine was being tested under laboratory conditions. Similarly, the implementation of a compiler program that claimed to allow recession could disable the function, or cause it not to work properly, depending on, for the instance, the date or time of day, the machine environment, or the particular iteration or count of the software execution.

He thus fails to distinguish between the design statement for a compiler that allows recursion, and the instantiation of that design in code. Moreover, no software is a perfect implementation, which causes the analogy inevitably to stumble. And by hinting at the notion of design in computer languages (what he signifies as the ‘trick’), Dawkins inadvertently undermines his analogy, since that notion of an architect has no role to play in evolutionary development, natural selection being an essentially haphazard process. Too many of his metaphors (for example, the arms-race, p 340; or ‘if we think of natural selection as a sculptor’, p 359) contain this notion of design at work, and thus weaken his whole argument, since the congenial atheist would assuredly deny the role of any ‘Designer’ in the process of language evolution. While many of the mechanisms by which genetic change occurs are still mysterious, that does not mean they are mystical. Following up on this theme, Dawkins later goes on to praise Chomsky’s idea of the language-learning apparatus being genetically implanted in the brain – which also strikes me as a bogus concept, since so many languages have implementations of syntax that are utterly antithetical and incompatible with other schemes. This is the weakest part of Dawkins’s theorizing.

Still, it was all a stimulating and enjoyable read, if you can put up with Dawkins continually reminding you how clever and successful he has been.

P.S. The New York Times informed me, on November 25, that the Saeed Book Bank in Islamabad, Pakistan, sells a thousand copies of Dawkins’s atheist treatise ‘The God Delusion’ each year. Not many people know that.

New Commonplace entries appear here.                                                                                                     (November 30, 2015)

4 Comments

Filed under Personal, Science, Technology

‘All The News That’s Not Fit To Archive’

We relational database people are well-organized, methodical. We like analysis and business rules, strong notions of identity , the use of sets and non-significant keys, normalized designs and value-based links, precise versioning and time-stamps, and careful promotion of systems into production, with secure fall-back procedures. All that is tech-talk, but it means something in the real world. (One of the first articles I had published, back in 1980, in Datamation, was titled ‘The Importance of Good Relations’, which showed the link between solid database design and flexible business practices.)

Yet the Web has changed all this. When I first developed my website, under Microsoft’s FrontPage, there was some semblance of a test environment and a production environment. I would develop the site on my computer, and when I was ready, and had made sure all the links were defined, and pointed to real pages, I would upload the whole kit and caboodle to the host site, where the new system would replace the old, giving me the option of importing all pages that had changed (but admittedly with no easy fall-back to the previous version). No more. I now use something called WordPress, which I invoke on a remote server. It allows me to compose and save drafts of individual pages, but it is otherwise tightly integrated with the production system. If I promote a new page, it goes live immediately, and if I change it again ten seconds later, the page is immediately replaced, with the previous one lost for ever. (Unless it found its path to some entity called the Wayback Machine, which is described in a fascinating article by Jill Lepore in the New Yorker of January 26, 2015, titled The Cobweb: Can the Internet be archived?)

I mention all this in connection with my last plaint from the January blog, about the New York Times, and its practice of making changes to its electronic versions of articles after they have been published in the printed version (or the late printed version, since that happens, too. We in North Carolina get an earlier version than the people up in New York, for example.) The reason this concerns me is primarily one of research integrity, since there is no longer a ‘paper of record’ on which historians can rely. I made this point in an email to the Public Editor, whose office eventually acknowledged my inquiry, promised to look into it, but then withdrew in silence. So, after a couple of weeks, I checked out the paper’s Statement of Standards and Ethics, and wrote to the Vice-President of Corporate Communications. The essence of my message ran as follows:

“For there is a vital question to be answered: ‘What is the paper of record?’ Your slogan on the first page of the printed edition is still ‘All The News That’s Fit To Print’, but apparently some of that news is Not Fit To Archive. What happens when historians attempt to use the paper for research purposes? Do they have to keep separate clippings files, since the electronic version is unreliable, and has been purified in some way for later consumption? Is there an active policy under way here that should affect your Ethics statement? How are decisions made to ‘improve’ the content of articles that have already appeared in the printed edition? Why are these not considered ‘Corrections’ that would normally be posted in the relevant section? How often does this happen?”

I received a prompt response, but it was all very dismissive and casual:

“The change you noticed was simply the result of normal editing, which takes place constantly for news stories, both between print editions and for successive online versions. In this case, additional information (including crowd estimates) was added to the story between the early print edition and the final print edition, which meant something had to be cut for the story to fit in the same space. In most cases, the final print version is the one that remains permanently on nytimes.com, though in some cases a story continues to be updated or revised online even after the final print edition.”

So I countered as follows:

“But I must state that I think that you (and I am not sure who ‘you’ are in this case) are being far too casual about this policy, simply treating the process as ‘normal editing’. Is there an audit trail? Do you keep all versions? What changes are allowed to be made after the final print version? Why cannot the on-line version (which has no size constraints) include all the text? Is there any period of limitation after which no further amendments can be made? How do you plan to explain this policy to readers, whose ‘trust’ you say you value so much?

I am sure you must be aware of the current debate that is being carried on in the world of academic research, where annotations to URLs in serious articles often turn out to be dead links instead of reliable sources. A Times ‘page’ no longer has a unique and durable identity, which I believe is an important issue.

I look forward to some deeper explanation of this policy in the newspaper.”

Well, maybe I should get out more. As Sylvia would suggest to me: “You clearly need something better to do.”  But I maintain that it is an important problem, not just concerning journalistic integrity, and getting the story right the first time, and not correcting quotations that the speaker wanted to withdraw (which we are told goes on).  It is more to do with what is known as ‘content drift’ and ‘reference rot’. As Jill Lepore’s article states: “. . .a 2013 survey of law- and policy-related publications found that, at the end of six years, nearly fifty per cent of the URLs cited in those publications no longer worked. According to a 2014 study conducted at Harvard Law School, ‘more than 70% of the URLs within the Harvard Law Review and other journals, and 50% of the URLs within United States Supreme Court opinions, do not link to the original cited information.” A more subtle problem is that the links may work, but the content may have changed  ̶  may have been edited, corrected, improved, revised, or sanitised. For researchers like me, this can be very annoying, as books these days frequently cite URLs rather than printed sources in their references, and when those pages do not exist, one feels cheated, and may also wonder whether they have been modified. The academic process has been debased. If one has text in the New York Times that is no longer on the archive, does it still exist? Is it still valid? Do I really have to maintain my clippings files, as opposed to an index of URLs? (To make her point, the Times Vice-President had to send me a scan of the two printed versions of the relevant page in question.)

We shall see. I haven’t received a follow-up to my second inquiry yet. Either the Times doesn’t believe it is an issue, or the managers there are having a big debate about the topic, which they don’t currently wish to share. I’ll provide an update if I do hear anything.

The normal set of Commonplace Updates this month. (February 28, 2015)

Leave a Comment

Filed under Media, Personal, Technology