Enigma and the Poles

Chris Smith (@spy_historian) tweeted a series of tweets about the Polish contribution to Enigma cryptanalysis this morning. I copy them here, adding some comments, and then write a bit more broadly about the issue. 

CS: Polish work on Enigma was truly impressive. They broke it while the British basically ignored it because they deemed it insoluble. A waste of time.

TC: The British had broken the commercial variant of Enigma early in 1927, and an improved solution was developed in time to exploit its use by the Spanish and Italian Navies during the Spanish Civil War and subsequently. I don't think it's fair to say that they thought the military variants insoluble, but up to 1939 they had no idea how to approach the problem. Put crudely, people who broke book-based codes, and simple electromechanical ciphers couldn't break more complex machine-based ciphers.

CS: They recognised the value of machine-based approaches. Though the Bomba was rapidly rendered obsolete by upgrades to Enigma systems, the Bomba was proof of concept. Use machines to break machines. I've argued elsewhere that Enigma and Bombas were an industrial revolution.

TC: Mechanical support to cryptanalysis wasn't new: Hollerith machines had been used in Room 40 during the First World War and other machines were being proposed in the second half of 1939 as GC&CS recruited from a deeper pool. The specific Polish breakthrough was to design, build and deploy a machine that mimicked Enigma. Following Turing's meeting in Parish with Rejewski he adopted the same concept for his (otherwise very different) Bombe.

CS: The contribution of the Poles, who shared their successes with the British, paved the way for upscaling that culminated in the SIGINT phenomenon that was #BletchleyPark. The BP Trust were right to create a Polish memorial, Prince Andrew was right to gift Poland an Enigma.

TC: Although the BP Trust were wrong to give one of the Enigmas GCHQ had loaned to them to Prince Andrew to gift to the Poles … !

CS: However, it has become a trope that this Polish contribution has been largely unrecognised in Britain and the US. My argument is that this is simply untrue. In fact, from the early days of Ultra becoming public knowledge - 1974 - the Poles were recognised.

CS: It is almost impossible to find a book that doesn't recognise their *massive* contribution. Yet these same texts often state that they have been under-recognised while recognising them. It is a weird, self-replicating myth. 

TC: The problem is that the nature, and particularly the source, of the Polish contribution was either ignored or grossly oversimplified (as largely was the case with the UK's own cryptanalytic work): so little was released by GCHQ in the first twenty years after the 1974 revelation that mythology filled the gaps. 

CS: So powerful is the myth, it has caused minor international spats. The Polish ambassador to the UK complained about Polish elision and misrepresentation (arguably rightly) in the 2001 film Enigma. In 2016, the Polish state commissioned a touring exhibition to correct the record.

TC: Both Enigma and The Irritation Game played their part in reinforcing the mythology (just as U-571 ignored the UK) but Hollywood blockbusters aren't documentaries.

CS: But the record didn't actually need correcting. The early British lit on BP/Ultra was clear that Poles did the lion share of early work. See: Lewin (1978), Calvocoressi (1979), Hinsley et al (1979), Collier (1982), etc.

------------------------------

Actually, Hinsley et al were wrong and their record did actually need correcting. Vol 3 pt 2 (1988), Appendix 30 (pp 945-939) was written by Joan Murray (née Clarke) and Henry Dryden, both wartime Siginters who had stayed on to work at GCHQ, and who were retained after retirement to update the version of the Polish contribution recounted in Appendix 1 of Vol 1 which was written on the basis of records and knowledge which was later proved – for example by the publication of Rejewski’s memoirs – to be incomplete. At best, I don't think it's unreasonable to have expected Hinsley et al to have done rather better first time.

CS: So where does this all come from? As ever Group Captain F.W. Winterbotham. His big splash book, The Ultra Secret (1974), which (sort of) revealed Ultra totally got the Polish work wrong. He was writing from memory and, besides, didn't know everything. Loads of that book is wrong.

CS: Yet as soon as it was published, key people in the know, not least Tadeus Lisicki, a wartime Polish intelligence officer and cryptanalyst, wrote to the papers, in 1974, to point out the Polish work. Lisicki compiled a dossier that formed the basis of important books by Poles.

TC: By Poles, in Polish, and while some were later translated, their impact was limited, not least because the authors weren't appearing at book festivals, on Radio 4 or in broadsheet review pages.

CS: Examples include Garlinski (1979), Woytak (1979) and Kozaczuk (1984). All of which have appeared in English. Newspapers, TV shows, radio comedies and even movies have made the point - though poorly in the case of Enigma. So why does this myth of anglo 'chauvinism' persist?

CS: None of this is to dispute the role Poles such as Marian Rejewski, Jerzy Różycki and Henryk Zygalski played. Quite the reverse. What fascinates me is the endurance of a myth that these pioneers have been elided from English language historiography. They haven't.

I remember telling a friend, when I became GCHQ's Historian-designate in 2008, that the two things I didn't see myself getting involved in were Enigma and VENONA. How wrong I was! Enigma remains a live issue for many – perhaps most – people outside the academic Intelligence Studies community who were interested in Intelligence history.

In the British public imagination the Bletchley Park story was the acme of the Boffinry: the British had been successful in the Second World War because they had out-thought their enemies. To the list of back-room boys coming up with Spitfire, Radar and bouncing bombs, was added the super-smart mathematicians who in complete secret helped win the war by breaking all German cryptographic systems. As with the Battle of Britain, Bletchley Park became part of a mythology of plucky little Britain fighting on alone and prevailing against all the odds, in spite of the evidence. The Polish contribution became merely transactional: a Pole handing over to the British an Enigma machine which was reverse engineered.

This popular narrative would not survive unchallenged the changes in Poland after 1989: the return to democracy, Poland joining NATO and the EU, and the move to the UK of younger Poles who had learned at least some of the story of Poland's contribution to the Enigma story. A Polish/UK Historical Commission reported in the first decade of the twenty-first century and described in English for the first time the breadth and depth of the Polish contribution to intelligence across the board, while GCHQ's massive 1994-2004 release of Second World War records had been absorbed and it was possible to begin to approach the question of Anglo-Franco-Polish cooperation on Enigma rather more reasonably than had been the case previously. Pioneering work by Dermot Turing and Marek Grajek also produced new accessible historical information that could be shared in English and Polish.

The problem I found as GCHQ Historian was that while 'the record' was becoming clearer, the Polish sense of their contribution having been slighted for so long had allowed a mythological counter-narrative to develop in which the UK and France would never have come near to solving Enigma without the Polish contribution. It was for that reason that I coined the term 'Enigma Relay' to try to make two points: first, that the credit of solving Enigma belonged to the allied team, the Poles, French, British and Americans each running separate laps and passing the baton; and second, that solving Enigma wasn't something worth 100 brownie points and that each of the four nations should scrabble to establish how many of the 100 they could each claim. International Intelligence cooperation doesn't work like that; solving Sigint problems doesn't work like that. I tried the 'Enigma Relay' out on a few people and then, when Polish, UK and French Sigint representatives agreed to meet in Warsaw in 2014 to celebrate the seventy-fifth anniversary of the tripartite meeting at which the three countries agreed to share all they knew, put the 'Enigma Relay' concept into the speech given there by Iain Lobban, Director GCHQ (https://www.gchq.gov.uk/news/director-gchq-commemorates-crucial-pre-war-enigma-information-sharing-meeting-poland).

Did this resolve all of the issues? No, of course not. Narratives and counter-narratives, and mythologies, have a life of their own and accurate history will always find it hard to compete against what non-historians would like a two dimensional 'truth' to look like. But reinforcing on every possible occasion the fact that success against Enigma took a lot more than a couple of very bright mathematicians thinking great thoughts, however crucial those great thoughts were, is part of the job. And making British audiences realise that most Sigint successes since 1939 are due to GCHQ's partnerships with Sigint agencies in other countries goes beyond mere intelligence history, and, hopefully, leads people to reflect that intelligence doesn't happen in a vacuum.

Sigint Communications (part 1)

This follows on from my previous post, in which I said, referring to a twitter discussion: 

'Part of agreeing with @jock_bruce here is a belief that environmental awareness is an essential precursor for Sigint, and that all is a waste of time without adequate Sigint Comms from intercept site to HQ. Sigint is about the whole of Comms.'

Sigint relies on having a dedicated, high-quality, high speed communications network, linking its national headquarters, intercept stations, intelligence allies and customers. High quality is absolutely essential; high speed will always be necessary in some parts of the network but not in others; a dedicated network is the outcome of two factors: the need for the highest levels of security across the whole of the network, and the need to ensure that prioritisation of traffic flow is something decided on by the Sigint agency and not by the organisation that supplies the comms infrastructure.

The first issue is to ensure that the headquarters can communicate with its intercept stations and with its customers. Preparing for the Second World War, one of the advantages of choosing Bletchley Park as the war station for GC&CS was the fact that the GPO had laid trunk cabling along the LMS railway. When the military elements of GC&CS returned to London after having deployed temporarily to Bletchley during the crisis of autumn 1938, they were able to make reasonable guesses about the comms capacity they would need in wartime, and by August 1939 BP had been well linked by teleprinters and telephones both to intercept stations in the UK and to Whitehall. This network of landlines was expanded and improved during the war under the aegis of the Y Committee. Overseas, however, underpinning the whole question of Sigint communications was the GC&CS policy of centralising UK cryptanalytic effort in the UK, partly from the increasing complexity of cryptanalytical problems, partly from the dearth of trained cryptanalytical staff if a large number of service centres were to be established overseas. (Later experience and the development of cryptanalytical machinery tended to confirm the original policy.) Three overseas cryptanalytical centres were in existence in 1939: the FECB for which GC&CS trained first naval and subsequently military officers; Sarafand, an Army commitment insofar as staffing was concerned, and Simla for which the Government of India found the personnel.

The Mediterranean campaign produced the CBME and the various commands in the field. The FECB was replaced by Delhi and Colombo/Kilindini on the entry of Japan which brought into existence the American organisations in Washington and the SWPA and the inter-allied Combined Bureau in Brisbane. The American policy was likewise to centre cryptanalytical work in Washington. Interception was spread right round the world and intercepted traffic had to be sent to the main centres as rapidly as possible. Long distance air mails were as a rule neither rapid nor certain enough as a principal means of conveyance, so that the whole or nearly the whole burden of carrying the signal correspondence of the Japanese Armed forces round the world fell upon Allied telecommunications. The intelligence produced from the intercept then had to be redisseminated by the same means.

The main source of material from GC&CS between the wars was Diplomatic correspondence which passed over the ordinary commercial WT services, for which interception in the United Kingdom was pure routine, or by cable, the lines of which in very many cases transited British territory. By special arrangements copies of all such Diplomatic correspondence were supplied to GC&CS, coming from overseas where required by sea bag. This was a leisurely proceeding but adequate for the Government' s purposes.

On the military side, and recognising that timeliness was likely to be a bigger issue for military Sigint than for diplomatic, the first meeting to investigate improvements in timeliness was held December 1937 but resulted in little other than an agreement that specified military intercept might be sent by air bag (under diplomatic protection) rather than by sea. The collection of DF bearings, however, needed a more real-time solution. 'C' wrote in April 1938 that 'cable companies claimed that the result of the Derby could be received in the uttermost corners of the earth within 5 seconds of the result being known' and that this must be mirrored by the Signal branches of the Services if DF was to be of any value. This didn't result in any specific action.

In September 1939 the problem of Y communications did not loom very large. The Army interception units that accompanied the BEF only had eleven intercept sets and ten DF sets between them and no plan was in place to site them near the network of telegraph and telephone lines established by the Royal Corps of Signals. The use of wireless for passing DF results was forbidden on security grounds. It was assumed that any Sigint would only be relevant to the BEF Staff so no dedicated link to the UK was planned. The RAF intercept unit sent to France was linked by WT to Cheadle, and thus, if necessary, with GC&CS but was regarded as an offshoot of Cheadle rather than an independent unit, since there was no Air GHQ in France until the end of January 1940. In other words no special system of communications for Y formed a part of the BEF pre-war plan, while in the case of the RAF the deployed unit was part of the home defence system. After a reorganisation of the Army Y system an I(s) staff was formed and sited near GHQ, as was 2 Company GHQ Signals, and a cryptanalytic party formed from the Military Section of GC&CS was sent to the French GQG. Smaller intercept units were moved to Corps HQs at Roubaix and Douai but the only communication with them was by bad telephone lines or by dispatch rider over singularly bad roads. I(s) was, however, in touch with MI8 by unreliable teleprinter though it was recognised that these links would only work while units were static. Material intercepted close to the Front could not be sent to GHQ in a timely manner, so I(s) missed many of the perceived advantages of being in the operational theatre rather than in the United Kingdom. The only means of any military intercepted traffic reaching the United Kingdom was by air bag, though a new DF station erected near Chartres was linked to Chatham by telephone and formed part of the Chatham DF network.

The first RAF unit at Fismes placed two DF units in Bar sur Seine and Amiens in order to form a baseline. These stations were linked by French Post Office landlines and Fismes was eventually linked to AI1(e) through the British controlled Rheims Central. This, it would seem, was led through to Leighton Buzzard Defence Teleprinter Network Switchboard and so, when working, gave them access to Cheadle and GC&CS (though all traffic has to pass en clair). The DF telephone lines connecting Bar sur Seine and Amiens with Fismes constantly broke down and when the organisation was later linked to the Intelligence Staff at AASF and BAFF the same occurred: 'our greatest handicap is landlines' reported the unit. The result was that Fismes seldom got more than a single line bearing, which made DF (and the DF units) essentially useless. A second RAF unit was formed to intercept Italian Air Force traffic from the south of France. This unit had no communications with GC&CS (which produced a lot of good IAF reports) and was handicapped by having few if any trained operators: it was of no real value. It used a courier service, French as far as Paris and British thereafter, which was expected to get the material intercepted through to GC&CS in 24 hours but, in fact, took about a week.

The Admiralty claimed to have a satisfactory teleprinter line to pass naval intercepts from the South of France and told AI1(e) who managed to obtain one, but only in May 1940. In the same way efforts to connect the Bar sur Seine station by British line to Fismes were only successful on the day when the station was handed over to the French during the retreat.

There were, however, arrangements for GC&CS communications with the French: in May 1939 it was agreed that Paris should telephone London at 3 p.m. each day and reports would be exchanged and confirmed later by bag. By August 1939 there was some level of communication between Bletchley Park and the Deuxième Bureau by RT, presumably conducted by Section VIII SIS. A courier service carrying cryptanalytical information continued and was extended to cover meteorological intercepts. The cryptanalytical party from GC&CS sent to the French GQG to collaborate on German Police ciphers and medium echelon Army traffic s needed no dedicated communications with the UK.

The 'Manual of Military Intelligence in the Field' current in 1940 stated that 'Wireless telegraphy is such a reliable and efficient medium of intercommunication that its use in war is indispensable to a modern army. Indeed it is likely that it may prove the only practicable method of signal communication in campaigns involving rapid movement over long distances' but in terms of Sigint nothing had actually been planned to make this a practical proposition.

There are several points to be made about this first phase of Sigint communications. First, nobody had thought about the problems of dealing with the dissemination of high echelon traffic that had to be worked at the centre. Second, there was no solution of German Enigma and no optimism in GC&CS that there might be, so there was no reason to think out a plan for disseminating it. When the Norwegian key was broken in April 1940 there were only normal signals channels and normal signals crypt systems to disseminate the intelligence. When the main GAF key was broken during the Battle of France it was far too late to institute a new communications plan.

The Army view of field Sigint was that it would be wanted in the field and they arranged for interception and for DF on a local scale. They foresaw 'rapid movement' by the enemy but not the corollary: that this would need rapid Sigint communications to cope with it. They had been slow to interest themselves in strategic DF and the development of the Chatham DF network was, at the outbreak of war, still a comparatively recent idea. Chartres DF was an outpost of this network and was given communications in conformity with the scheme. But that the same principle applied to the field units seems to have been overlooked. Very much the same may be said of the RAF field organisation.

Lastly, the Battle of France was over so quickly that there was nothing to inform planning for better communications: either between units; or between units and the deployed Intelligence Staffs; or between any deployed unit and the centre in the United Kingdom. Given the overwhelming defeat of the Allied Armies it would have been hard to show that lack of Sigint communications had in any way affected the issue or that the contribution of Sigint to the general pool of intelligence had been other than comparatively meagre. So when the focus of the war moved to the Mediterranean, Sigint as a whole was still lacking in experience of its real requirements for either long distance communications or with Command Headquarters or for field communications between Y units, and between them and any base organisation formed in the rear.