Sigint Liaison with Finland

 

The UK now has a Defence Agreement with Finland and Sweden, and will soon be formally allied to both countries in NATO. It is not well known that for a short period in the Second World War the UK and Finland found themselves on the same side: our enemy's enemy was our friend and in spite of the alliance between Nazi Germany and the USSR, German occupation of Poland, Denmark and Norway, the Soviet occupation of the Baltic States and the part of Poland the Germans had agreed to let them have, and Swedish neutrality, the UK and Finland enjoyed a Sigint partnership for just over a year.

British Sigint had had relationships with the French, Italians and Americans during the First World War. These had all disappeared by the time GC&CS was formed on 1 November 1919. RAF Sigint did its own deal with the Estonian military Sigint authorities, and for most of the 1930s equipped and trained the Estonians in return for Estonian intercept logs of traffic of the Soviet Air Force, though this ended in 1939. In the 1930s, under the auspices of SIS a relationship was developed with the French to discuss Enigma, and from this grew the 'Enigma Relay', the exchange of information that led to Enigma becoming an exploitable cipher system.

This meant that GC&CS had become receptive to the idea that foreign Sigint services could be met as partners and early in 1940, this led to the idea that somebody should visit Finland to see whether the Finns were doing anything against the Soviet Union, the ally of Nazi Germany, that might benefit UK Sigint. At this point there was no dedicated UK effort against the Soviet military: an interservice cryptanalytical section would not be set up (in Wavendon, near Bletchley) until the summer of 1940.

Tiltman, the Head of the Military Section, went to Finland to meet the Finnish Sigint organisation, a meeting brokered by the SIS representative in Helsinki and the UK Military Attache. He arrived in Finland on March 1st 1940 and stayed for ten days most of which he spent with Major Hallamaa, the head of Finnish Army Sigint, at Kerava. Hallamaa showed Tiltman the whole of his organisation and expressed the willingness of the Finnish General Staff to cooperate fully in Sigint matters with the British.

The Finns were willing to exchange their knowledge of Russian codes and ciphers in return for the radios they were very short of. They needed 50 receivers, 6 DF sets and 6 high-powered transmitters to link their units in the field. The War Office in London confirmed that some of the equipment was available and the Finns handed over a forty page handbook on Russian Army and Naval communications.

The equipment was released on 15 April 1940, and the Finns were further informed that the UK would fund the Finnish purchase of further radio equipment in Sweden up to the value of £500. Some equipment arrived in Helsinki but when another GC&CS officer arrived in Finland to develop liaison on the Russian target he found that the Finns had only obtained 4 receivers, though a search while he was there produced three of the DF sets as well, and the Finns appeared satisfied that the British were acting in good faith. They agreed to provide Soviet intercept provided the British reciprocated: in the Finnish system the clerical effort involved would have to be justified to higher authority by a quid pro quo. They gave the UK the current encryption system being used by the Soviet Baltic Fleet as well as military systems which had become mainly readable. In return GC&CS offered its work on encryption systems used by the Soviet Navy Black Sea Fleet which was being taken in Palestine.

The exchange of intercepts was not straightforward. The air route between the UK and Finland was only open during the summer months, and the Finns were unwilling to open a new two-way radio link to the UK so the only way was to send the traffic by air to Stockholm from Helsinki in a diplomatic bag and to forward the traffic from there on commercial wireless circuits and vice versa. This would have worked if a Typex machine could have been sent to Stockholm and if sufficient Typex capacity had been installed at Sarafand to deal with the Black Sea traffic. However, there were significant problems in the supply of Typex, and by March 1941 Denniston argued that if only GC&CS could be allocated 10 lbs weight in the Stockholm daily bag the need for Typex would disappear, but neither Typex nor the 10 lbs of freight became available.

In spite of all the assurances it had given, the War Office did not actually send the rest of the radio equipment originally promised. It took until summer 1941 before this was sorted out and by then the situation had changed completely. The Germans declared war on Russia and a German mission was sent to Finland just as the final decision to send the missing sets was taken.

The last signal from Finland contained an assurance that they would not reveal to the Germans their Sigint cooperation with the UK and asked for an assurance that the British would not tell the Russians of Finnish cryptanalytical success against Russian systems. This assurance was given, and the Finns and the British each kept their promise (something perhaps easier for the British side than for the Finnish).

GC&CS developed its links with the US, and maintained its links with France and the exiled Poles until December 1942, but this marked the high water of non-Five Eyes cooperation. The Finnish experience had shown GCHQ that a buccaneering attitude to foreign relationships might bring results, but always at a potential cost of the partnership being prey to other considerations, and the dispersal of French and Polish cryptologists after the Nazi occupation of Vichy led to a new and cautious approach which would last for many years.

Sigint Liaison with the Russians

After the German invasion of Russia in June 1941 a military mission (No 30 UK Military Mission) was sent to Moscow and learned that the Russians had a Y Service. In August, 1941 a Y officer from the Middle East went to Moscow. He was given little information, and wasn't allowed to see anything of the Soviet organisation. However, as British relations with the USSR had been bad for many years this was considered unsurprising, and in October 1941 the War Office on its own initiative attached an Army officer (who had been working as a Sigint traffic analyst) in 30 Mission. At the same time the Admiralty negotiated permission to build a Naval Y Station in the far north aimed at providing early warning of any attack on the convoys planned to bring supplies to the Russians. On the Army and Air Force side it was hoped principally to obtain Russian-intercepted German traffic and some help if possible on Japanese systems. In fact, apart from the supposition that Russian Sigint would be less efficient than British there wasn't any clear policy, other than that allied intelligence liaison was a "good thing". 

The Admiralty received Soviet permission to erect a Y Station at Polyarnii, near Murmansk, and British permission to give the Soviets information about Auka (a low-grade Luftwaffe system), and, in May 1942, sent a naval officer from GC&CS to Murmansk to arrange for the exchange of Naval and TA intelligence. As a result of this some naval intercepted traffic reached GC&CS and there was also an interchange of DF bearings.

The War Office representative also made some progress. He was authorised in February 1942 to train the Russians to break the German police ciphers which were being used in rear areas on the Russian front, as well as showing them some traffic intercepted in the UK. (It was never the intention that British success against Enigma should be revealed.) He reported that this had produced a favourable reaction and he asked to be allowed to expand the exchange, giving the Russians identities recovered from callsign analysis of certain German stations in return for traffic intercepted in Russia. In June 1942 he was recalled for discussion of policy and went back to Moscow as representative, not of the War Office, but of the Y Board. But the mood in Moscow had changed: the Russians stalled and eventually in November the Head of the Soviet Army section dealing with Traffic Analysis suggested that cooperation should be discontinued.

Naval cooperation in the far north continued on a restricted basis, though some intercepted material inaudible elsewhere did reach the UK. This lasted until spring 1943 when, just as the UK staff were preparing to hand over another batch of Y material, the Russians suddenly entered the British site, sealed up the transmitters and took away the "meaconing" apparatus (a means of falsifying German Beacon transmissions.) At the same time it came to British knowledge that the German Y Service was reading some high echelon Russian cipher systems without much difficulty. Convoys were not actually in operation at this time, but were expected to resume and the closure of the station and Soviet insecurity greatly increased the hazards of running them.

In return for German Police material, the UK had given the Soviet military the Japanese Military Attaché code, the "Bird book" and index (the GC&CS process for breaking out German enciphered callsigns), the solution to German aerodrome serviceability reports, the process for recovering German police keys, Auka, and some ISOS keys. The Russians had also been made aware that the UK could read the Army tank code, Army medium grade field ciphers as well as the cipher used by one of the German sabotage units. The UK side decided that unless and until the Russians resumed cooperation, no further material should be sent. There was no point in keeping a representative in Moscow and the liaison officer, while retaining the title 'Russian Liaison Officer', was relocated to GC&CS. GC&CS had only one vote on the Y Board but it was a significant one. its attitude towards Russian cooperation was becoming more assertive and more reluctant. 'Jumbo' Travis wrote to Heads of Sections in May 1943 saying that he was prepared to consider an exchange, but only if there was a solid return "as I am not now a believer in Russian cooperation".

Although it had been decided that the Y Board should have no representative in Moscow, 'C' felt that an SIS liaison officer might usefully represent Sigint as well. One went to Moscow in the summer of 1943, just in time for several things to happen all at once. The Soviets asked for a full description of the German Enigma machine; they captured a copy of Auka themselves; and German Y Service traffic intercepted by the UK revealed that Russian ciphers were still being read by the Germans.

A meeting took place in summer 1943 after the British Ambassador made representations to Molotov, and it became clear that the Soviets' main interest was to find out what exactly what the British evidence of Russian cipher insecurity was. They were told that the information had come from a disaffected Austrian prisoner of war and from information from Japanese Military Attaché decrypts; they were also told that the UK side would give no further Sigint-derived information until the Russians delivered on their promises of the previous year. There were a few sporadic meetings and it became perfectly clear in spite of the SIS representative's attempts to put the best possible interpretation on what was going on, that the Russians were trying to get what they could from the British without giving anything away themselves. Telegrams continued to pass desultorily throughout 1944 and an Enigma machine and a small amount of very low level Y information was sent, though probably more for SIS's benefit than for Sigint's. Before the end of the war cooperation had ceased.

Why were the Soviets so reluctant to carry out an exchange of material? GCHQ believed post-war that the Soviet Sigint organisation had made the authorities in Moscow aware that the material being offered by the UK was low grade and that therefore either the UK was keeping its aces up its sleeve, or it didn’t have any capability worth wasting Soviet time on.

In my last post I told the story of how some information derived from Enigma decrypts had been published after having been passed by military censors. Commenting on Twitter @jock_bruce said that "in that story the point is that the folk clearing stuff for publication need to know what they’re actually protecting". I think that there is a similar issue here: those making decisions – the single services, and subsequently the Y Board – didn't understand that any half-way competent Sigint organisation would realise that valuable information was being withheld.

Sigint liaison worked with the French and the Poles when there was a full exchange of knowledge between the national Sigint organisations of each country, and subsequently worked with the US and Australia, Canada and New Zealand, because there was full sharing and integration of staff in each other's organisations. It couldn't work if the exchange was partial and low-level, and it certainly wouldn't work if non-Sigint personnel were representing Sigint.

 

Locking a Stable Door

There is an interesting story in Guy Liddell's diaries (The National Archives (TNA) KV 4/187 and 4/189) about publication of Sigint secrets in a book published in 1941.

March 8th (1941)

SIS are in a great state about the publication of a book called The Diary of a Staff Officer, who (sic) in addition to criticising the French and British Commands in rather an outspoken manner makes reference to work affecting the GC&CS. I communicated first with Mr Ray of Methuens, the publishers, and then with Dick's brother who is a partner in the firm. 1,900 copies have already been issued and since further demands are being made by the booksellers, most of them must be in the hands of private individuals. Meanwhile, another 7,000 copies are in preparation and are to go out on Tuesday next. Dick's brother tells me that the book was submitted to the MOI and passed by all 3 service representatives before Xmas. It was then published serially in the USA. It is possible however that the offending passages may have been omitted. A few copies have gone to Canada and the book has been extensively reviewed here in the Press. I am suggesting that the 7,000 copies should not be circulated and that the censorship and D.4 should be warned against letting any of them out of the country. I feel that if we do more than this we shall only be drawing attention to what we are anxious to donceal (sic). We shall be in a better position to decide on Monday.

March 10th (1941)

I saw Stewart and Hopkinson about the officer's diary. We discussed the problem in all its aspects and thought that before coming to a definite conclusion it would be wise to see Gribble at the earliest possible moment.

March 11th (1941)

Stewart, Lennox and I saw Gribble at the WO. He admitted that he knew the messages came from a very secret source but he thought he was amply covered by having submitted his diary for censorship. He admitted however that he had shown the document previously to his agent to see whether its contents were like (sic) to interest the public if they were published in book form. It was explained to him that certain passages constituted a grave indiscretion and that it was in the public interest that he should cooperate with us in doing anything possible to pick up the pieces. We eventually agreed that even though the offending passages had appeared in the Saturday Evening Post it would be better to issue the new edition in amended form. Gribble was to tell his agent that he wished to make certain amendments in the text and therefore that he would like the return of the proof copy from America before any further action is taken. As regards the Manchester Evening News who were asking permission to publish the book in serial form, it was decided to see first whether they were proposing by their own selection to exclude the offending passages. If not, we should have to say that the book could not be published serially until certain amendments had been made which the author considered desirable in the light of subsequent information received.

…

In the afternoon Dick's brother came down and we discussed details about Gribble's book. It was apparent that if the offending passages were taken out it would be quite clear that a new one had been gummed in. We therefore decided to buy up the whole of the second edition which was already bound and have arranged that the offending page in the other two editions should be removed altogether, with those which were joined to it. These would be collected before the new amended pages were delivered to the printer.

2^nd February (1942)

According to an intercepted letter a New York publisher is trying to arrange with Watt, London, for the publication of Major Gribble's book Diary of a Staff Officer in German. It is to be printed in Switzerland. Lennox has got hold of Gribble and will tell him to stop the publication. If the book circulates in Germany there is a possibility that what has been overlooked may come to light.

(Key to people: Dick is Dick White, MI5 officer and future DG; Stewart is Stewart Menzies, 'C'; Hopkinson is probably Henry Hopkinson, PS to PUS FCO (subsequently Lord Colyton); Lennox is Gilbert Lennox, MI5 officer and responsible for MI5 liaison with the War Office.)

Fascinated by this story, I managed to locate and buy a copy of the first (offending) edition of this book. (The second edition was, presumably, bought up and destroyed; and there were at least third and fourth editions according to abebooks.)

The offending passages appear to be two of the entries for 12 June 1940. Here is the entry for the whole of the day.

June 12th. 1130 hours. The German have three fresh army corps South of RHEIMS. This town was occupied yesterday when the French made a complete withdrawal to the MARNE. There are now 80 German divisions on the CHANNEL – MONTMEDY front. The French possess some very exhausted 35 divisions. There is no longer any hope of success for the French.

PS (private sources) is providing a fund of information. There were 45 messages last night.

1200 hours. The last of these messages consisted of a single word: "SEDAN". This is the German word for bombing PARIS.

1210 hours. It was repeated to us again today that the lack of enthusiasm in the French armies could be attributed to the fact that one third of them was drawn from the occupied territory. Even more, it is thought that at least half the soldiers in the French army have now lost their homes through German occupation.

2300 hours. Very unpleasant news tonight. A big break through over the SEINE with enemy reported as far over as DREUX and EVREUX. Looks like the next move towards the SEINE.

It is pretty clear that Major Gribble was trying it on. He knew how sensitive the material originated from Sigint was, and he knew that the volume of reportable intercepted messages was significant. But he guessed that the military censors wouldn't know as much as he did. I imagine that the fact that "SEDAN" was identified as a codeword message (CWM) meaning "Bomb Paris" was what caused the stir when the book was published, but it's hard to believe that the Germans hadn't assumed that the CWM would be compromised as soon as Paris was bombed.

This diary entry was written before the most severe levels of security were designed for the protection of Sigint, and it is in part because of what happened in France that things were tightened up. As de Grey says, in Appendix III of his Organisation and Evolution of British Sigint (TNA HW 43/78):

The policy adopted in regard to the Enigma decrypts had no repercussions during the "Sitzkrieg" [the Phony War] – they were dull and of no immediate operational value. It was not until the Battle of Norway that the intelligence became of such operational value that its dissemination in a wider field became a problem. Norway was over before any organisation could be set on foot. The Battle of France supervened but it was not until May 22nd that the current German Air Force key was solved and thenceforward solved regularly – that is until a week or two before the end of the Battle. The arrangements to supply GQG [Grand Quartier General: the French GHQ] with results were untried and neither Ally was versed in the exchange of such material. GHQ and AHQ had not had the experience of receiving a flood of such material, were in no position to interpret its significance and were in ignorance of its reliability. The whole front was crumbling and although it is possible that some of the information reached the higher commands it is doubtful whether it was of any use. The War Office channels of dissemination were abandoned possibly for fear of their insecurity and SIS WT was substituted in the last few days. The difficulties of secure dissemination to the commands in the field were thus fully brought to light.

The diary entry illustrates de Grey's point: as well as copious and detailed predictive intelligence from intercepts, Gribble's office was handling top level information about German and French Orbat and German movements, as well as engaging in dubious analysis of what might or might not be demotivating the French Army. This isn't a picture of a well-functioning Intelligence Staff: those involved had no training in or experience of what they were supposed to be doing.

The solution chosen to solve the Sigint part of this problem was ULTRA and the special handling procedures associated with it were effective in protecting the fact of successful cryptanalysis and the products which that success brought, but at a cost: not everybody who handled intelligence had access to all relevant intelligence on a given subject. The level of clearance needed to have access to the most sensitive intelligence was granted to only a small proportion of members of intelligence staffs. That in turn produced problems of its own.

 

 

 

 

 

 

 

The Yangtze Incident And A Do-It-Yourself One Time Pad

The story of HMS Amethyst and the Yangtze Incident used to be well known, but is less so today. Briefly, during the Chinese Civil War HMS Amethyst was sailing up the Yangtze from Shanghai to Nanking to provide support to the British Embassy to the Chinese Nationalist government which was temporarily based there. The British frigate was fired on by Chinese communist forces and severely damaged, and was forced to anchor for ten weeks, every attempt to move provoking renewed fire from the communist batteries. On the night of 30 July 1949 Amethyst made a daring and successful 100 mile dash down the Yangtze and rejoined the fleet. there is a little known cryptographic footnote to this story. 

After being attacked, HMS Amethyst had prepared to scuttle herself and had destroyed all crypto materials. From comments made by the local communist commander during protracted negotiations, it was clear that his forces were intercepting the plain language traffic between Amethyst and CinC Far Eastern Station, so how could Amethyst and the CinC plan her escape?

The Chief Naval Signal Officer Far East (Cdr C R Williams RN) came up with a plan to create a One Time Pad (OTP) which would then be used to re-encipher message sent using the Government Telegraph Code, a copy of which was held by Amethyst but which was also known to be in the possession of the communists.

The OTP was created as follows: in a first column, the surnames of all RN officers and men on board were listed in alphabetical order.

Column two listed the first name of the next of kin of each of the people in column one.

Column three listed the placename in the address of the people in column two as given by the person in column one.

Column one was now discarded.

Working down column two, letters were converted into four figure groups – so JOAN became 1015 0114. Groups were written across the page, with five four figure groups on each line. Once column two was complete, the same process was used with column three.

This pad was only used for one message. In it, Amethyst was told the times, frequencies and indicator groups of a series of dummy messages which would be addressed to CinC FES on the Fleet broadcast. Amethyst should copy these messages and use the groups transmitted to make 'In' and 'Out' pads. She was also told how to use the pads without a codebook.

During the afternoon of 30 July Amethyst sent a Flash signal: "DTG 300657Z July. Top Secret. CinC info Concord from Amethyst. I am going to try and break out two two zero zero item repeat two two zero zero item tonight three zero July Concord set watch eight two nine zero." (HMS Concord was a destroyer sent up the Yangtze to support Amethyst's escape.)

A lot of people still think that the security that OTPs offer is based on their randomness: in fact it's based on their unpredictability. In this context, the Chinese communists' knowing how the single use OTP was constructed introduced significant weaknesses. Apart from the fact than an agent in London might have been able to access the list of crew and their next of kin, column three was likely to have a disproportionate number of repeats of "Portsmouth", "Plymouth" and "London". But even if they had been able to do break and read this one message, it is likely that there would have been a time lag, and during that period they would have had to have copied and saved every single message on the Fleet Broadcast, to have been able to reconstruct the OTPs built from the dummy traffic being broadcast.

I would like to be able to say that after a carefully carried-out risk assessment an expert panel decided that the risk was worth taking, but there is no record to suggest that this was the case. It is unlikely that we'll ever know, but my guess is that this was a piece of local ingenuity, which happened to turn out well.

Ten avoidable problems which made the Royal Navy's encryption exploitable in 1939

By the time of the outbreak of war in 1939, Nazi Germany had thoroughly penetrated Royal Navy encryption systems, and the Navy's changing its main codebook caused barely a hiccough to German Navy cryptanalysts: in fact it took until 1943 to secure RN and allied naval encryption and during this time the German Navy destroyed millions of tons of allied shipping and came close to preventing the supply of food and military supplies to the UK from North America. Just as Bletchley Park exploited poor security by the German military in their use of encryption, the German Navy's B-Dienst was able to exploit the poor security of the codes used by the Admiralty. How could this have happened? A note written years later by Captain D A "Willie" Wilson RN identifies areas where the Admiralty got it wrong. I think the issues they raise are as relevant today as they were between the wars, because although security today addresses technological challenges that our forebears couldn't have imagined, the foundations of security and the mindset of the security practitioner are fundamentally the same.

The Admiralty got things so badly wrong because there was no single coordinating body managing Comsec in the UK, and no recognisable centre or standard of Comsec expertise. GC&CS had a responsibility to provide advice on Comsec to civil ministries, and could be approached by the service ministries if they so wished. Within GC&CS, just as Comint was the responsibility of Denniston, the Head, Comsec was the responsibility of Travis, the Deputy Head. However, Travis was also responsible for: collection management, for the reporting, indexing, and distribution of intelligence reports, for the GC&CS Registry, for liaison with SIS, and, from 1938, for liaison with the service ministries. All of this was in addition to his responsibility (to 'C', interestingly: not the Head of GC&CS) for the British Codes Section.

The fundamental problem was that nobody questioned the principle that good cryptanalysts would make good communications security advisors, when in fact their advice would be limited by the extent of their cryptanalytic skills. In a non-mechanical environment, this meant cryptanalysts saying "this is the sort of code I can't break", and assuming a) that that meant that nobody else would be able to, granting that code an invulnerability; and b) given that cryptanalysts didn't do traffic analysis, that crypt security was the only security that mattered: that wireless security (secure traffic management) wasn't something that needed to be considered.

1. Don't split responsibilities among different people.

The Director of Naval Intelligence (DNI) was responsible for technical and physical security, but was dependent on technical cypher matters on the advice given by GC&CS. The Director of the Signal Department (DSD) was responsible for providing the means of communication, and for providing the Coding Staff, but he had no responsibility for wireless security, (ie for denying the enemy intelligence from traffic analysis). The Paymaster Director General (PDG) had responsibility for the provision and training of cypher staff, but had no responsibility for the cyphers used.  In the Fleet the Secretary to a Flag Officer was the Squadron or Fleet Cypher Officer, responsibly for seeing the proper use of cyphers afloat. Within the Admiralty M Branch of the Secretariat was responsible for the distribution of Cyphers and Codes through the Navy, and the Secretariat provided the Admiralty cypher office, known as "War Registry".

2. Choose your advisers carefully

GC&CS was required to advise (but not mandate) security to the services between the wars. In fact Travis advised the Admiralty, Tiltman advised the War Office, and Josh Cooper advised the Air Ministry, but they didn't talk to each other about security, and didn't consult any of the rest of the cryptanalytic staff in GC&CS. Their advice was, therefore, strictly limited. Cipher security was the subject of a lecture on the Accountant Officers Technical Course (a prerequisite for a Secretary to a Flag Officer) but by the beginning of the 1930s this was a lecture about cryptanalysis given by Bodsworth and Knox, and was unlikely to have been illuminating. The Long Signal Course for officers specialising in communications had no lectures at all on communications security.

3. Know when to ask for a second opinion

The key piece of advice that the Admiralty needed was about the security of the 'Long Subtractor', the key used to encipher messages once encoded. In effect, this was a one time pad which was reused. How many times could it be reused without compromising security? Theoretically, it should never be reused, but in practice, could it be? Wilson wasn't happy with the response: once gave absolute security, twice was almost guaranteed, three times wasn't really dangerous, but more than three was, and using the same piece of key five or six times was positively dangerous. This was very bad advice, and Wilson knew it, but there was no move to change advisor (nor any obvious other advisor to turn to).

4. Security doesn't come second to intelligence

The Admiralty had two billets in GC&CS for officers: the idea was that this would be a source of officers with practical experience in security. But everyone in GC&CS knew that the department's main job – its important job – was intelligence production, and the officers posted in became, for the most part, cryptanalysts. Intelligence breakthroughs are of tremendous value: if your security is no good, it will be your adversary who makes the intelligence breakthrough by exploiting your communications.

5. Don't do the enemy's job for him

The RN used three different basic codebooks: Tactical Code (3 letter groups, not reencrypted); Administrative Code (five figure groups, reencrypted with a Long Subtractor); Naval Cypher (used for operational traffic: four figure groups, reencrypted with a  Long Subtractor). This meant that anybody intercepting a naval message immediately knew, without decryption, what sort of message had been sent. And although different Long Subtractor tables were used for different regions and commands, the indicator – the explanation of the start point in the relevant table – wasn't disguised at all. The German Navy's cryptanalysts had it easy.

6. Don't assume before you understand

In order to gauge how many of each reciphering table to print and distribute (a job that would always be complex given the Navy's worldwide extent) the Admiralty had to make assumptions. It assumed that the proportion of administrative to operational traffic in wartime would be around 80:20 – they were wrong: it was the other way round. If the Royal Navy officers in GC&CS had been doing the job for which they had been posted in, they would have seen this by looking at traffic levels in Italian Naval material collected for GC&CS during the Abyssinian Crisis and the Spanish Civil War.

7. Even if it ain't broke it might still be worth fixing

According to Wilson, by 1937 when Typex, the UK version of Enigma, came into service, Lord Mountbatten was pressing for it to be used by the Royal Navy but got nowhere because of the lack of a single person able to resolve to bring a new encryption system into use. Lord Mountbatten concluded that it was just too difficult, but even putting Typex on a few ships in the North Atlantic could have made a tremendous difference. I asked the Duke of Edinburgh about this in 2013 and he confirmed it. (That is probably the most outrageous name drop you'll see today.)

8. Who needs what?

As a matter of routine, Commands, shore establishment and major ships were given copies of all reciphering tables, so that anybody could communicate securely with anybody else. But why (for example) would CinC Nore need to communicate securely and directly with the Senior Naval Officer Upper Yangtze? Why couldn't the few (I imagine) messages between most territorial commands not be relayed through the Admiralty. Apart from the difficulty of printing and distributing so much key material, it meant that if a ship was believed to have potentially been captured by enemy forces, all key material on board had to be assumed to be compromised.

9. Understand the vulnerability of your mode of communication as well as of your crypto

After Wilson had been Head of a new section NID10/DSD10 in the Admiralty early in the war, he set up a new section to look at the way the Navy communicated to see whether the Germans might be able to derive intelligence from analysis of the traffic. This had never been done before.

10. Security versus Operability

Security and operability don't always have to be in competition: designers of systems need to understand how users communicate to design practicality alongside security; system administrators need to ensure that the reasons for the rules they impose are understood; and users mustn't try to subvert the rules and compromise security simply to make life easier. In order to make Royal Navy comms practice secure against German traffic analysis, DSD10/NID10 introduced a series of security measures without explaining them to the signalmen who had to implement them. As a result, they were widely ignored and subverted. When this was discovered the signalmen asked for a senior officer to investigate the matter. Admiral Somerville (who had been in charge of wireless comms for the Mediterranean Fleet during the First World War) was asked to do this and he came down firmly on the side of security, but was able to explain why to the signalmen, from whom there was no further opposition to the new measures.

Sir Edward Bridges and the Development of UK Communications Security

I think I'm a bit closer to an explanation of something that has exercised me for a few years: why did Sir Edward Bridges, first as Cabinet Secretary and then as Permanent Undersecretary (PUS) at the Treasury, take such an interest in Communications Security (Comsec), from 1941 until his retirement in 1956? He made sure that in January 1944 the creation of a Cypher Policy Board independent of the JIC was approved by the Chiefs of Staff and the War Cabinet. This meant a better defined Comsec organisation in GC&CS with the authority to mandate security standards across the whole of UK Government. In 1954 he drove through the institutional separation of the LCSA (what would eventually be called CESD, then CESG, then turn into the NCSC) from GCHQ, a separation that lasted until 1969.

His first experience of encryption probably came in March 1929, when, as a Treasury civil servant, he became a member of the Inter-Departmental Committee on Cypher Machines, on which he sat alongside Edward Travis, Deputy Head of GC&CS and its lead for Comsec. The committee's report seems to have been ignored: it recommended the "O'Brien-Gardiner machine" no working model of which was available. Enigma was turned down on the grounds that it was of foreign manufacture.

Eventually, Wing Commander Lywood RAF improved the on the design of Enigma and came up with Typex which was first used operationally in 1937. GC&CS was kept away from the development of Typex by either Lywood, or the RAF, or both: they didn't believe that the advice being provided by GC&CS on security was adequate: he (or the RAF, or both) was right. By the time of Dunkirk in 1940, Typex was the only British military encryption system the Germans couldn't read. (I will write more about this catastrophic failure another time.)

In October 1941 four cryptanalysts from Bletchley Park wrote to the Prime Minister complaining of delays in supplying the manpower needed for GC&CS's mission. Churchill's response was immediate: his famous 'Action This Day' minute in which General Ismay was instructed to "Make sure they have all they want on extreme priority and report to me that this has been done". In fact it looks likely that while Ismay worked with 'C' on the manpower required for Bletchley Park, Bridges interested himself in the security mission, at the time mainly farmed out to Mansfield College Oxford, with a small nucleus at Bletchley. (Certainly, some months later, when GC&CS approached the Treasury to increase its establishment across the board, its letter said "if you need more evidence I can only refer you to Bridges or Ismay" (letter quoted by Nigel de Grey in HW 43/76).)

Bridges understood that the primary reason for the failure of British Comsec was due to a systemic issue: GC&CS's charter did not allow it to mandate security standards for the armed forces, only to provide advice. A separate issue, but which contributed to the overall failure, was that the quality of the advice was poor, and, anyway, wasn't always accepted. Bridges spent fifteen years godfathering a new structure for UK Comsec, which resolved both of these issues.

The Admiralty had sponsored an "Inter-Service Committee on the Security of Codes and Cyphers" early in 1941, and Travis and Tiltman represented GCHQ at its first and only meeting. It set up a "Technical Sub-Committee" chaired by Travis with three members: from the Admiralty and Air Ministry, and Tiltman, for GC&CS and the War Office. It met regularly (23 meetings between February and October 1941) and it changed its title to Cypher Security Committee. The parent body was replaced by a new "Inter-Services Cypher and W/T Security Committee" and it adopted the Cypher Security Committee, widening its membership beyond GC&CS and the three services to include Civil Departments (Foreign Office, Colonial Office, India Office, Dominions Office, Ministry of War Transport, Ministry of Food, Ministry of Supply, Postal and Telegraphic Censorship, Ministry of Aircraft Production and Ministry of Information) though these only attended by invitation when matters relevant to them were to be discussed.

This Committee was still advisory (though I don't think that its conclusions and recommendations were ignored or rejected by anybody), and had no forward planning responsibilities, but Bridges was still keen to see a more formal arrangement. He persuaded the original committee members to come up with a paper outlining the problems faced by the committee and after detailed discussion in November and December 1943 a paper was agreed by the Chiefs of Staff in January 1944 and adopted by the War Cabinet. Its main conclusions were:

1.            A new body called the Cypher Policy Board was created under the authority of the Prime Minister and the War Cabinet. Its membership was: 'C' (Chair) as Director-General GC&CS; the Cabinet Secretary (Bridges); Director General Signals Air Ministry (representing the Chiefs of Staff); Director GC&CS (Travis); Secretary (Communications Security Advisor GC&CS). It was responsible for: (a) deciding questions of policy governing the security of British cyphers, including decisions about new cyphering devices proposed and safeguards necessary for their use; (b) ensuring that the use of cyphers by the Services and by Government Departments was properly supervised; (c) advising on the security of cyphers used by Allies where appropriate.

2.            A new section of GC&CS was established to deal with the security of British cyphers and of Allied cyphers (insofar as British commands were concerned). This section was headed by a GC&CS Assistant Director with the title Communications Security Advisor (CSA) (Captain Wilson RN).

3.            The Cypher Security Committee became responsible to the Cypher Policy Board, and the Secretary of the Cypher Policy Board became its chair.

Bridges became PUS at the Treasury but took his membership of the Board with him. Even if communications could only command a small amount of his attention as PUS and Head of the Home Civil Service, he would nevertheless make time for it.

Some years later in 1952 the Admiralty complained about problems caused by the way in which the Cypher Policy Board had developed a committee structure. The matter was taken up at Board level and the Admiralty said that it felt that many of the tasks being performed by the Board might be better carried out by the Sigint Board. Bridges answered that the Cypher Policy Board had been set up in 1944 to see that the right cryptographic policy was devised, both from the standpoints of security and practicability and to ensure that this policy was carried out. He believed that it would be quite inappropriate for these responsibilities to be handled by the Sigint authorities. An officer from the Signals Division of the Admiralty carried out an investigation in conjunction with a Treasury civil servant nominated by Bridges.

The investigation looked at the structures of the Board but, surprisingly, also recommended the creation of a new agency, the Telecommunications Security Agency, which would subsume the Comsec responsibilities of GCHQ for planning and policy for new cryptosystems and speech security devices, and for their design and engineering development. It would also take on the responsibilities for Comsec which were still in control of the services. GCHQ and the service ministries were persuadable: the military complained that existing structures usually saw GCHQ representatives asking for solutions which gave 100% security but were practically unusable; to which GCHQ countered that no new off-line system had been developed since the war because of the military's insistence on continually changing the specifications for new machines. Bridges persuaded them not only of the need for a single body but also that both GCHQ and the services would accept that the new Agency's Director would sit above them and act as arbiter.

In the event, LCSA – the London Communications Security Agency – was set up in 1954. It shared the central London office block on Palmer Street which GCHQ had acquired in 1953 for a 'front office' and maintained the bulk of its staff at Eastcote when the rest of GCHQ went to Cheltenham. The divorce was never final: LCSA (soon renamed CESD) was too small to be an independent agency, and GCHQ continued to provide administrative and estate services; the mathematicians needed by both organisations continued to form a common pool; and particularly after the retirement of Bridges, CESD had no real voice in Whitehall or ability to face the services down where necessary other beyond GCHQ's ability to speak on its behalf.

Trying to work out why Bridges had interested himself to such an extent is difficult. When the NCSC was being set up, and I had been through this tale with Ciaran Martin, its first Head, he asked Lord Bridges if there were any papers held by the family, or any family folk memory on this subject: there weren't.

I think that Bridges, having decided to look out for Comsec while General Ismay did the same for Comint after the 'Action This Day' minute from Churchill in 1941, realised that the subject needed higher price help than it was likely to get if he didn't look after it himself. His view (in modern language) was that Comint professionals are the wrong people to make Comsec policy, even though the input from Comint is the primary tool at the disposal of Comsec professionals. It is clear that he believed that if Comsec and Comint were housed in a single organisation, Comsec would necessarily suffer if its autonomy could not be maintained, and so arranged for it to separate from GCHQ as completely as possible.

Separation from GCHQ simply didn't work, so CESD came back to Cheltenham as CESG: but its autonomy was part of the deal, even if the degree of autonomy was a moveable feast. The NCSC model – "part of GCHQ" but headquartered outside Cheltenham – was a response to the radically different requirements cybersecurity demanded of both Comint and Comsec, but has finally achieved Bridges' vision.

Intelligence Officers' Small Talk: Tiltman Meets Liddell

 

Guy Liddell, the Director of Counter-Espionage in MI5 during the Second World War (and beyond) kept a diary – in fact he dictated an account of each day's activity which was typed up. The diaries from 1939 to 1953 have been released to The National Archives: they are an important resource for anybody interested in UK intelligence history. The entry for 24 October 1944 (TNA reference KV 4/195) recounts a conversation with John Tiltman, the "Chief Cryptographer" at GC&CS, what today would be called the Head of Cryptanalysis. They had each been appointed CBE in the 1944 New Year's Honours List and attended their investiture on 24 October.

"I attended the investiture at Buckingham Palace where I met Tiltman of GC&CS. Tiltman reminded me of the time when he and I had gone down to some firm in Southampton Row to inspect a holorith [sic] machine. This was a good many years before the war. He said he had no idea at that time that the holorith was going to be such a big factor in the work that he would be doing. He did not know what future there would be for GC&CS but he could not help thinking that the difficulties of the work might become insuperable when foreign govts. realised the mistakes that they had made during the war. Intelligence might well be driven back on the old cloak and dagger lines. Tiltman is now Chief Officer on cryptography and a Brigadier. He said that this does not prevent him from a certain amount of daily exercise in his special line without which he thinks his usefulness to the organisation would disappear in a very short time. He says that methods change so quickly that it is absolutely essential to do a certain amount of hack work, otherwise one's usefulness will entirely disappear."

There were very few contacts between GC&CS and MI5, even on operational matters: all communication was expected to go through SIS, and SIS decided what each needed to know about the other. This is the only record, I believe, of Tiltman dealing with a member of MI5 during the war and the conversation, as recorded by Liddell, looks like the sort of small talk two senior members of different agencies might exchange at such an event. But effectively Tiltman says only three things, and doesn't really tell Liddell anything.

Hollerith machines were extremely important at Bletchley Park, as they had been in Room 40, as a means of sorting and analysing information, but they were not the sharp end of Bletchley's information technology. By this time Bletchley's cryptanalytic work was being supported by some 200 Bombes (as well as sharing time on US Bombes) and by 5 Colossus machines (a sixth would come into service four days later). It is instructive that Liddell obviously got no sense of this.

The statement that cryptanalysis might become impossible once foreign governments became aware of the mistakes they had made and that therefore Humint would become dominant is superficially plausible but it begs the question of how they were going to find out. Sigint security had developed into a sophisticated process during the war, with even the fact of interception of enemy communications not being discussed publicly. The Ultra dissemination system was designed to restrict knowledge of the fact of successful cryptanalysis of enemy communications to an absolute minimum, while an equally sophisticated process allowed action to be taken on Ultra intelligence by finding plausible alternative sources for it. The success of the US attack on Japanese encryption would become known during the Pearl Harbor enquiry, but that was a year after this conversation.

Tiltman's views about the need to keep his hand in if he is to remain useful to GC&CS is uncontentious, though I imagine he really meant that he enjoyed any opportunity to be a cryptanalyst instead of an administrator.

The main surprise to me in this conversation is that the subject of signal security didn't come up. At this time MI5 was trying to establish itself as lead department in this field even though it controlled none of the assets required either to monitor UK service or civilian traffic or to secure communications. A Wireless Telegraphy Security Committee had been in existence since 1941 but MI5 had never been a member, and MI5 raised the subject at the JIC – which GC&CS wasn't a member of. Liddell's diaries show that MI5 was not impressed by the Radio Security Service, whose job was to intercept and process illicit signals (messages sent by unauthorised individuals using radios) and which was controlled by SIS. (It would become part of GCHQ after the war for a short period before its functions became part of GCHQ normal business.) Reading Liddell's diaries, it feels to me that MI5's instinct was that signal security was part of national security, and therefore MI5's business, even if it had no realistic way of securing signals.

This is a really frustrating extract: there was so much that Liddell and Tiltman could have talked about, but they didn't. Two senior members of two secret agencies recognised each other at a formal state occasion and exchanged pleasantries, but nothing else.