Signals intelligence in modern history

SIGINT is a contraction of SIGnals INTelligence. Before the development of radar and other electronics techniques, signals intelligence and communications intelligence (COMINT) were essentially synonymous. Sir Francis Walsingham ran a postal interception bureau with some cryptanalytic capability during the reign of Elizabeth I, but the technology was only slightly less advanced than men with shotguns, during World War I, who jammed pigeon post communications and intercepted the messages carried.

Flag signals were sometimes intercepted, and efforts to impede them made the occupation of the signaller one of the most dangerous on the battlefield. The middle 19th century rise of the telegraph allowed more scope for interception and spoofing of signals, as shown at Chancellorsville.

Signals intelligence became far more central to military (and to some extent diplomatic) intelligence generally with the mechanization of armies, development of blitzkrieg tactics, use of submarine and commerce raiders warfare, and the development of practicable radio communications. Even Measurement and Signature Intelligence (MASINT) preceded electronic intelligence (ELINT), with sound ranging techniques for artillery location. SIGINT is the analysis of intentional signals for both communications and non-communications (e.g., radar) systems, while MASINT is the analysis of unintentional information, including, but not limited to, the electromagnetic signals that are the main interest in SIGINT.

World War I

Radio communications were fairly new at this time. At the strategic level, nations gained access to commercial undersea communication cable traffic. Tactically, wired telephones were in wide use, and techniques of intercepting them through ground returns were developed. These intercept techniques have had a resurgence in later wars, where radio was less available or impractical. On the declaration of war, one of Britain's first acts was to cut German undersea cables, forcing them to use radio, which the British could intercept. The destruction of more secure wired communications, to improve the intelligence take, has been a regular practice since then. While one side may be able to jam the other's radio communications, the intelligence value of poorly secured radio may be so high that there is a deliberate decision not to interfere with enemy transmissions.

Russia, when preparing for the Russo-Japanese War in 1904, had established a tradition of poor communications that would last well after the fall of the Romanovs. The success of this and related salvage and rescue work persuaded the Russian Navy to install wireless sets on many of its ships. In early 1904, the Russian fleet prepared for war with Japan. The British almost immediately began to intercept their communications, with the complaint “An intelligence report on signals intercepted by HMS Diana at Suez shows that the rate of working was extremely slow by British standards, while the Royal Navy interpreters were particularly critical of the poor standard of grammar and spelling among the Russian operators.”^[1] Failure to properly protect its communications fatally compromised the Russian Army in its advance early in World War I and led to their disastrous defeat by the Germans under Ludendorff and Hindenburg at the Battle of Tannenberg. Similarly, the interception and decryption of the Zimmermann Telegram was an important factor in the US decision to enter the War.

Radio researchers at the British Marconi Company realized that strange signals they were receiving were German naval communications, and brought them to the Admiralty. Soon, the British were operating a network of listening posts called "Y-stations", with Admiralty Room 40 doing the traffic analysis and cryptanalysis.^[2] In World War II, the British referred to their traffic analysis function as the "Y service".

In contrast, battles have been lost, or not fought, when senior commanders asked the traffic analysts and direction finders the wrong question. In World War I, someone at the Admiralty knew a little too much detail about SIGINT without fully understanding it. He asked the analysts where call sign "DK" was located, which was that used by the German commander when in harbour.^[3] The analysts answered his question precisely, telling him that it was "in the Jade River". Unfortunately the High Seas Fleet commander used a different identifier when at sea, going so far as to transfer the same wireless operator ashore so the messages from the harbour would sound the same. The misinformation was passed to Jellicoe commanding the British fleet, who acted accordingly and proceeded at a slower speed to preserve fuel. The battle of Jutland was eventually fought but its lateness in the day allowed the enemy to escape.

Jellicoe's faith in cryptographic intelligence was also shaken by a decrypted report that placed the German cruiser SMS Regensburg near him, during the Battle of Jutland. It turned out that the navigator on the Ravensburg was off by 10 miles (16 km) in his position calculation. During Jutland, there was limited use of direction finding on fleet vessels, but most information came from shore stations. A whole string of messages were intercepted during the night indicating with high reliability how the German fleet intended to make good its escape, but the brief summary which was passed to Jellicoe failed to convince him of its accuracy in light of the other failures during the day.

France had significant signals intelligence in World War I. While the key intelligence achievement in blunting the German drive on Paris in June 1918 was the cryptanalysis of Georges Painvin, had French intercept personnel not captured the message in the ADFGVX cipher, there would have been nothing to cryptanalyze.

Between the World Wars

There was substantial SIGINT work between the World Wars, although the secrecy surrounding it was extreme. While it was primarily COMINT, ELINT emerged with the development of radar.

Both sides developed direction-finding (DF) and communications interception stations during the war, although those programs often began with naval search & rescue.

Canada

Canada's first signals intelligence intercept site, Special Wireless Station #1, was built in 1939, in Ottawa. "#2 SWS was located at Grande Prairie, Alberta and #3 SWS at Victoria, British Columbia. Victoria also had a remote high-frequency direction finding (HF/DF) site in Nanaimo approximately 60 miles (97 km) to the northwest."^[4] 1 Canadian Special Wireless Group (1CSWG) deployed to Australia in January 1945.

Germany

By the mid-twenties, German Military Intelligence Abwehr was intercepting and cryptanalyzing diplomatic traffic. Under Hermann Göring, the Nazi Research Bureau (Forschungsamt or “FA”) had units for intercepting domestic and international communications. The FA was penetrated by a French spy after 193????, but the traffic grew to a point that it could not easily be forwarded. In addition to intercept stations in Germany, the FA established an intercept station in Berne, Switzerland. It penetrated most cryptosystems other than the UK and US.^[1]

German Condor Legion personnel in the Spanish Civil War ran COMINT against their opponents.

United Kingdom

After the First World War, the British Army and Royal Navy signal intelligence efforts were combined to form a new organization, reporting to the Admiralty, called the Government Code and Cypher School (GC&CS), with Alastair Denniston as its first head.

While it was operational in 1919, it was realized most of its current work was diplomatic, so it was transferred to report to the Foreign Office in 1922. Both GC&CS and the Secret Intelligence Service reported to Hugh Sinclair, in London. In May 1927, Prime Minister Stanley Baldwin made public some GC&CS solutions of Soviet intercepted message, causing a massive Soviet cryptographic change.

By 1940, GC&CS was working on the diplomatic codes and ciphers of 26 countries, tackling over 150 diplomatic cryptosystems.^[5]

United States

Naval direction finding and message interception

US communications monitoring of naval signals started in 1918, and continued, but was used first as an aid to naval and merchant navigation. In October 1918, just before the end of the war, the US Navy installed its first DF installation at its station at Bar Harbor, Maine, soon joined by five other Atlantic coast stations, and then a second group of 14 installations.^[6] These stations, after the end of World War II, were not used immediately for intelligence. While there were 52 Navy medium wave (MF) DF stations in 1924, most of them had deteriorated. The Navy transferred, in July 1941, the remaining stations to the US Coast Guard.

As tension with the Japanese grew, the COMINT situation was being reviewed. In the early 1930s, the Navy started implementing high frequency DF (HF/DF). Eleven locations were planned, primarily on the Atlantic Coast, and beginning with Bar Harbor, Maine, early 1936. The first operational intercept came from what would later be called Station CAST, at Cavite in the Philippines. There were still technical problems, a development program started, and the first advanced station created at Winter Harbor. In July 1939, the function turned from training and R&D to operations, and the Navy officially established a Strategic Tracking Organization under a Direction Finder Policy.

By December 1940, the Navy's communication organization, OP-20-G, had used HF/DF on German surface vessels and submarines. Training continued and cooperation with the British began. In April 1941, the British gave the US Navy a sample of their best HF/DF set from Marconi.

All remaining navigational DF stations were transferred to the Coast Guard in May 1941, and the Navy concentrated its efforts on COMINT, reporting to OP-20-G under Commander Laurance F. Safford. By December 1941, the Navy established a strategic HF/DF and intercept station, with Atlantic, Pacific and West Coast net control stations managing 20 sites. Increasingly, new site selection emphasized COMINT value over HF/DF. The prototype intercept station had been in Maine, initially in Bar Harbor but relocated to Winter Harbor in 1935. It principally intercepted European traffic to Tokyo, but also had a section intercepting Soviet traffic. Intercept reorganization came during the first week of September, with Atlantic stations reemphasizing HF/DF, and interception at Jupiter, Florida and Cheltenham, MD. The Cheltenham station was replaced by Chatham MA as the primary intercept station.^[6]

Ship platforms

Beginning in 1937, US naval ships started intercepting communications, beginning with the destroyer USS Hatfield (D-231). It anchored at La Rochelle-Paris and started operating, but lack of intercept training quickly became evident. The Director of Naval Communications established policies and procedures; it should be noted that COMINT reported Communications, not Intelligence.

The European squadron, 40-T, with USS Raleigh (CL-7) as flagship, originally was assigned to evacuate civilians from the Spanish Civil War, but a secondary COMINT duty became evident, with a unit established on the USS Omaha (CL-4) in 1938, soon designated as Station F, intercepting German, and Italian traffic, later in the Mediterranean.

Station F moved to the new flagship, USS Trenton (CL-11) in June 1939. They noted significant communications changes two days before the German invasion of Poland, and the intelligence significance was noted and forwarded to Washington. In 1939, the Atlantic was the priority, with a very short belief Japan was not a threat. In 1940, formal liaison began with the British, under the terms of a highly secret policy accepted in 1937. The Special Naval Observer in London was the point of contact, and formal COMINT exchange began in November 1940.

Station F, still on ships, concentrated on Italian traffic in 1940. OP-20-G began cryptanalytic work in July 1940. OP-20-G was acutely aware of British ship losses in the Battle of the Atlantic. Focus remained on the Atlantic, with Agnes Driscoll, the chief cryptanalyst under Lieutenant Lee W. Parke, worked on Italian systems; German system remained secure.

Increased pace in 1941

In 1941, the sensitivity of COMINT increased when the US gave Britain a Purple cipher machine. The British provided technical information on Cryptanalysis of the Enigma as well as current British HF/DF procedures but not, at this stage, the full derived Ultra intelligence.^[7]

Operational priority increased when Winter Harbor and Amagansett received teleprinter messages for faster relay of intercepts to the analysts in Washington. While the emphasis was now on Japanese traffic, other traffic of interest was still studied.

COMINT against the inter-war Japanese

COMINT of Japanese traffic proved invaluable to the Allies at the Washington Naval Conference in 1921, through cryptanalysis by Herbert Yardley. Then-Secretary of War Henry L. Stimson closed the US Cipher Bureau with the words "Gentlemen do not read each other's mail." Luckily for US COMINT, the Army offered a home to William Friedman after Stimson closed the Yardley operation.^[8]

Friedman's team had four analysts that would become bright figures in American cryptology: Solomon Kullback, Frank Rowlett, and Abraham Sinkov. Kahn's memorable comment "If Yardley was the star of American cryptology, Friedman was the Sun" remains apt. They developed largely manual cylindrical and strip ciphers, but, as a result of Friedman's advances in cryptanalysis, machine ciphers became a priority, such as the M134, also known as the SIGABA. While the SIGABA was a rotor machine like the German Enigma machine, it was never known to be cracked. It was replaced by electronic encryption devices.

SIS, in contrast with Yardley's dependence on cable companies, set up its own radio intercept organization. Eventually, the training and intercept functions were separated for both administrative and security reasons, when, a centralized signals intelligence unit, the 2d Signal Service Company, was set up at Fort Monmouth on 1 January 1939 to control all Signal Corps personnel at the permanent monitoring installations. In this period, SIS paid less attention to tactical SIGINT.

SIS Intercept Stations, 1939

Station	Primary role	Secondary role
W (Winter Harbor)	Italian Naval	Axis Diplomatic
M (Cheltenham)	German Naval	Axis Diplomatic
G (Amagansett)	Diplomatic	None
J (Jupiter)	Diplomatic	None

World War II

A true world war, SIGINT still tended to be separate in the various theaters. Communications security, on the part of the Allies, was more centralized. Given that there were three major Axis powers, each primarily operating in a subset of the theaters, it is convenient to look at SIGINT from a primarily theater standpoint. From the Allied perspective, the critical theater-level perspectives were the Ultra SIGINT against the Germans in the European theater (including the Battle of the Atlantic, the Mediterranean Theater of Operations, and MAGIC against the Japanese in the Pacific Theater and the China-Burma-India theater. Germany enjoyed some SIGINT success against the Allies, especially with the Merchant Code and, early in the war, reading American attaché traffic. Japan was the least effective of the major powers in SIGINT. In addition to the official Allies and Axis battle of signals, there was a growing interest in Soviet espionage communications, which continued after the war.

British strategic stations were located at places including Darwin, Australia, and a Russian site. Major postwar stations include RAF Menwith Hill and Cyprus.

Allied European Theater

The use of SIGINT had even greater implications during World War II. The combined effort of intercepts and cryptanalysis for the whole of the British forces in World War II came under the code name "Ultra" managed from Government Code and Cypher School at Bletchley Park. By 1943, such was the extent of penetration of Axis communications and the speed and efficiency of distribution of the resulting intelligence, messages sometimes reached allied commanders in the field before their intended recipients. This advantage failed only when the German ground forces retreated within their own borders and they began using secure landline communications. For this reason, the "Battle of the Bulge" took the allies completely by surprise.

British Secret Intelligence Service

Initially targeting German spies in Britain, the "Radio Security Service" was soon intercepting a network of German Secret Service transmissions across Europe. Successful decryption was achieved at an early stage with the help of codes obtained from the British XX (Double Cross) System that "turned" German agents and used them to misdirect German intelligence. The combination of double agents and extensive penetration of German intelligence transmissions facilitated a series of highly successful strategic deception programmes throughout WWII.

Royal Navy

Early on, Admiralty dismissal of SIGINT information (also traffic analysis in this instance) contributed to the loss of HMS Glorious in 1940./

Perhaps the most important role SIGINT played for the Royal Navy, and the merchant ships it protected, was in the Battle of the Atlantic. By comparison with the close and garrulous radio communication between the U-boat submarine high command, BdU, and German submarines in the Atlantic, US submarines in the Pacific were as free as fish. While Ultra cryptanalysis certainly played a role in dealing with German submarines, HF/DF and traffic analysis were complementary.

It is unclear why the German submarine command believed that frequent radio communications were not a hazard to their boats, although they seemed confident in the security of their Enigma ciphers, both in the initial three-rotor and subsequent four-rotor versions (known as Triton to the Germans and Shark to the Allies). There was an apparent, mutually reinforcing belief that wolfpack attacks by groups of submarines were much more deadly than individual operations, and confidence the communications were secure. Arguably, the Germans underestimated HF/DF even more than they did British cryptanalysis.^[9] Apparently, the Germans did not realize that the Allies were not limited to slow, manually operated direction finders, and also underestimated the number of direction finders at sea.

Battle of Britain

ELINT and electronic warfare became critical parts of the Battle of Britain. R.V. Jones was a key scientist in the "Battle of the Beams", defeating Nazi radio navigation systems (e.g., Knickebein). While the Ultra COMINT successes against the Germans were not declassified until 1975, Winston Churchill paid homage to electronic warfare, and its companion ELINT, in his series on the Second World War:

During the human struggle between the British and the German Air Forces, between pilot and pilot, between A.A. batteries and aircraft, between ruthless bombing and the fortitude of the British people, another conflict was going on, step by step, month by month. This was a secret war, whose battles were lost or won unknown to the public; and only with difficulty comprehended, even now, by those outside the small scientific circles concerned. ... Unless British science had proved superior to German, and unless its strange, sinister resources had been effectively brought to bear in the struggle for survival, we might well have been defeated, and, being defeated, destroyed.^[10]

In modern terms, MASINT was as important as SIGINT in defeating Nazi navigational systems, with radar control of the defenses a key part of the Battle of Britain.

French Resistance and Free French

France consolidated a number of general intelligence and SIGINT units in World War II, producing the wartime Directorate of Studies and Research (DGER) by November 1944. As the Cold War heated, France was concerned with the presence of Communist networks among these units, so, in 1946, created the External Documentation and Counterespionage Service (SDECE) subordinated to the prime minister.^[11]

Efforts at US coordination during World War II

During the Second World War, the US Army and US Navy ran independent SIGINT organizations, with limited coordination, first on a pure personal basis, and then through committees. Perhaps the strongest outside effect, prior to and during WWII, was the United States Department of State and the White House, the only consumers of intelligence outside the military, especially since both the Army and Navy wanted to have the prestige of providing them with diplomatic COMINT. Note that while the Office of Strategic Services was a fairly autonomous WWII agency, it still, technically, reported to the Joint Chiefs of Staff and received COMINT through military channels.^[12]

During the war, the military departments became concerned with the creation of new cryptanalytic units in the US government, including the Federal Bureau of Investigation (FBI), Federal Communications Commission (FCC) and Department of State. The military finally formalized the sharing of targets in 1944, but that did not cover the non-military organizations. They established a Joint Army-Navy Radio Intelligence Coordinating Committee, which soon changed its name to the Joint Army-Navy Communications Intelligence Coordinating Committee.

US Army

After the Normandy landings, Army SIGINT units accompanied major units, with traffic analysis as or more important than the tightly compartmented cryptanalytic information. Bradley's Army Group, created on August 1, 1944, had SIGINT including access to Ultra. Patton's subordinate Third Army had a double-sized Signal Radio Intelligence Company attached to his headquarters, and two regular companies were assigned to the XV and VIII Corps.

The 3250th Signal Service Company, attached to V Corps, moved 10 times in June and July 1944, and suffered nearly 20 percent casualties during the Battle of the Bulge, including four killed in action.^[13]

US Navy

In World War II anti-submarine warfare (ASW), shore or ship-based SIGINT often vectored long-range patrol aircraft to U-boats, which they might detect visually or by airborne radar if the submarine was surfaced, or by early sonobuoys used from 1944 on, which could cue dropping depth charges or very early homing torpedoes. The Army demonstrated feasibility of the AN/CRT-1 sonobuoy, and, by 1944, the Navy had ordered almost 60,000^[14]

A daring US Navy feat that received very mixed reviews was the capture of the German submarine U-505 by Captain Daniel Gallery's escort carrier group. While useful cryptomaterial was taken from the boat, Gallery and his immediate chain of command were unaware of the Ultra successes against German submarines. There was considerable concern at Bletchley Park that if the Germans realized a U-boat, and presumably its Enigma had been captured, the Germans might change cryptosystems. The notoriously hot-tempered Chief of Naval Operations, FADM Ernest J. King considered court-martialing Gallery, but relented and authorized the award of a Distinguished Service Medal with a classified citation. Gallery and others had been ordered to not capture any German warships at that particular time, just before the invasion of Normandy lest the resulting change of cryptosystems leave the Allies blind. This was the first time since the War of 1812 between Great Britain and the United States that the United States had captured an enemy ship in wartime. To keep Germany ignorant of the capture of the U-505, the submarine was towed to Bermuda.

Appropriately, however, the first US sailor, Lieutenant Albert David, to go down the hatch of the submarine, which might have scuttling charges about to detonate or have water rushing in, received the Medal of Honor. The two sailors behind him received the Navy Cross.

Axis European Theater

The entire Nazi system suffered from Hitler's deliberate fragmenting of authority, with Party, State, and military organizations competing for power, with only Hitler really pulling the strings. Hermann Göring also sought power for its own sake, but was much less effective as the war went on and he became more focused on personal status and pleasure.

German air intelligence, during the Battle of Britain, suffered from the structural problem that subordinated intelligence to operations. Operations officers often made conclusions that best fit their plans, rather than fitting conclusions to information.^[15]

In contrast, British air intelligence was systematic, from the highest-level, most sensitive Ultra to significant intelligence product from traffic analysis and cryptanalysis of low-level systems. Fortunately for the British, German aircraft communications discipline was poor, and the Germans rarely changed call signs, allowing the British to draw accurate inferences about the air order of battle.

A 1939 German intelligence study^[15] discounted British radar and ground-controlled interception, and believed the only serious defenses were in the London area. Göring was not receptive to dissenting views that key targets were out of bomber range, and significantly out of the range of escort fighters.

Allied Pacific Theaters

Several theaters were involved in this part of World War II: CINCPAC/CINCPOA, CINCSWPAC, CINCCBI.

Allied cooperation in the Pacific Theater included the joint RAN/USN Fleet Radio Unit, Melbourne (FRUMEL), and the Central Bureau which was attached to the HQ of the Allied Commander of the South-West Pacific area.

Australian Army

After consultations between Australian and US signal and communications senior staff, MacArthur ordered Central Bureau to be created, partially to avoid his being dependent on Navy SIGINT.^[16] Central Bureau was made up of:

• The intelligence section of the former No. 4 Australian Special Wireless Section
• Australian Military personnel
• RAAF personnel
• US Army intelligence personnel who had escaped from the Philippines
• US Army intelligence personnel from USA (6 officers and 8 men of the 837 Signals Service Detachment)
• British intelligence staff from Singapore

At first, Central Bureau was made up of 50% American, 25% Australian Army and 25% Royal Australian Air Force (RAAF) personnel, but additional Australian staff joined. In addition, RAAF operators, trained in Townsville, Queensland in intercepting Japanese telegraphic katakana were integrated into the new Central Bureau. Other components of Central Bureau included:

• the Geographical Section which produced maps and geographical data about the SWPA
• the Allied Translator and Interpreter Section (ATIS) which interpreted millions of captured documents, intercepted messages and interrogated thousands of Japanese POW's
• the Australian Coast Watching Service
• a POW interrogation center.

Central Bureau broke into two significant Japanese Army cryptosystems in mid-1943.

Royal Australian Navy

FRUMEL was the joint US-Australian naval SIGINT unit. Commander, later Captain, Eric Nave did not stay long with FRUMEL, which was put under U.S. Navy control in mid-1942. He was sent to Central Bureau in mid-1942, but it has been suggested he dealt only with lesser Japanese systems, although he had both Japanese language skill and experience with their cryptosystems. The major systems were the target of US Col. Abraham Sinkov.^[17]

Until Central Bureau received replacement data processing equipment for that which was lost in the Philippines, as of January 1942, U.S. Navy stations in Hawaii (Hypo), Corregidor (Cast) and OP-20-G (Washington) decrypted Japanese traffic well before the U.S. Army or Central Bureau in Australia. Cast, of course, closed with the evacuation of SIGINT personnel from the Philippines.

US Navy

US strategic stations targeted against Japanese sources included Station HYPO in Hawaii, Station CAST in the Philippines, station BAKER on Guam, and other locations including Puget Sound, and Bainbridge Island.

US COMINT recognized the growing threat before the Pearl Harbor attack, but a series of errors, as well as priorities that were incorrect in hindsight, prevented any operational preparation against the attack. Nevertheless, that attack gave much higher priority to COMINT, both in Washington DC and at the Pacific Fleet Headquarters in Honolulu. Organizational tuning corrected many prewar competitions between the Army and Navy.

Perhaps most dramatically, intercepts of Japanese naval communications^[18] yielded information that gave Admiral Nimitz the upper hand in the ambush that resulted in the Japanese Navy's defeat at the Battle of Midway, six months after the Pearl Harbor attack.

US Army

The US Army had shared, with the Navy, the Purple attack on Japanese diplomatic cryptosystems. Many histories assume Purple included Japanese military cryptanalysis, but those were separate projects, although generally under the same organizations.

After creation of the Army Signal Security Agency, the cryptographic school at Vint Hill Farms Station, Warrenton, Virginia, trained analysts. As a real-world training exercise, the new analysts first solved the message center identifier system for the Japanese Army. Until Japanese Army cryptosystems were broken later in 1943, the order of battle and movement information on the Japanese came purely from direction finding and traffic analysis.

Traffic analysts began tracking Japanese units in near real time. A critical result was the identification of the movement, by sea, of two Japanese infantry divisions from Shanghai to New Guinea. Their convoy was intercepted by US submarines, causing almost complete destruction of these units.^[13]

Army units in the Pacific included the US 978th Signal Company based at the Allied Intelligence Bureau's secret "Camp X", near Beaudesert, Queensland south of Brisbane.^[19] This unit was a key part of operations behind Japanese lines, including communicating with guerillas and the Coastwatcher organization. It also sent radio operators to the guerillas, and then moved with the forces invading the Philippines.

US Army Air Force

Even as the planes burned at Clark Field, hours after the Pearl Harbor attack, Lieutenant Howard Brown, of the 2nd Signal Service Company in Manila, ordered the unit to change its intercept targeting from Japanese diplomatic to air force communications. The unit soon was analyzing Japanese tactical networks and developing order of battle intelligence. He moved from Manila to Corregidor on Christmas Eve.

They learned the Japanese air-to-ground network was Sama, Hainan Island, with one station in Indo-China, one station near Hong Kong, and the other 12 unlocated.^[13] Two Japanese naval stations were in the Army net, and it handled both operations and ferrying of aircraft for staging new operations. Traffic analysis of still-encrypted traffic helped MacArthur predict Japanese moves as the Fil-American forces retreated in Bataan.

Evacuated, as were most SIGINT people, from the Philippines, Brown, helped build the Australian-American intercept station, and 126th Radio Intelligence Company, at Townsville, Queensland. He later trained the Air Force SIGINT staff. US Air Force Far East, and its subordinate 5th Air Force, took control of the 126th in June 1943. The 126th was eventually placed under operational control of U.S. Air Force Far East in June 1943 to support 5th Air Force. Interception and traffic analysis from the company supported the attack into Dutch New Guinea in 1944.^[13]

The US began airborne ELINT against Japanese radar in the Aleutians, using a modified B-24 aircraft in January 1943. ELINT was much less significant in the early Pacific War than in the European Theater, probably because strategic bombing using electronic navigation aids was not a critical issue.

US Marine Corps

In 1943, the US Marines organized the 2nd Radio Research Platoon, which was the original unit in a chain of tactical SIGINT units that also made strategic contributions.^[20]

Japanese SIGINT

Japan had been fighting in China and Manchuria since the 1930s. They were overconfident in their communications security.^[3]

Indochina

In September 1940, the Japanese moved into the Haiphong area of French Indochina, claiming they wanted to disrupt supply lines to their war in China. In June 1941, they expanded their occupation to all of the colony, to which the US responded with embargoes that the Japanese regarded as a casus belli for the Battle of Pearl Harbor. US Army and Navy cryptanalysts were able to follow events, initially through their penetration of the RED cryptomachine, and then the Purple system, introduced in 1939 and broken in 1940.

Principally to track shipping, the US monitored Japanese, and eventually French colonial administration, traffic, through WWII. In general, the area was not of strong operational interest to the Allies, except for planning submarine attacks on shipping, and occasional air raids on transportation infrastructure. On the strategic level, however, the US began to learn more about the resistance groups in Indochina. These groups, especially the Viet Minh, fought the Japanese, but would later fight the French administration, and eventually the Republic of Vietnam (RVN; South Vietnam).

In March 1945, the Japanese, through their own COMINT, were alerted of a potential French coup against the Japanese occupation. Within 48 hours, all the French administrators and troops were captured, except for about 4,000 troops who fled into China.^[21]

September 1945 found an emboldened Viet Minh, under Ho Chi Minh and assisted by a US OSS team under MAJ Archimedes Patti, declare the independence of the Democratic Republic of Vietnam (DRV; North Vietnam).^[22] The Allies, however, did not recognize Ho's government, staying loyal to the French.

While the French claimed Ho's movement was Communist, US State Department analytic reports in 1947 and 1948, written from all-source intelligence including COMINT, gave no indication that the Vietnamese Communist Party was controlled by Moscow.^[21]

Western counterespionage

From 1943 to 1980, the Venona project, principally a US activity with support from Australia and the UK, recovered information, some tantalizingly only in part, from Soviet espionage traffic. While the Soviets had originally used theoretically unbreakable one-time pads for the traffic, some of their operations violated communications security rules and reused some of the pads. This reuse caused the vulnerability that was exploited. Venona gave substantial information on the scope of Soviet espionage against the West, but critics claim some messages have been interpreted incorrectly, or are even false. Part of the problem is that certain persons, even in the encrypted traffic, were identified only by code names such as "Quantum". Quantum was a source on US nuclear weapons, and is often considered to be Julius Rosenberg. The name, however, could refer to any of a number of spies.

Aftermath of World War II and the 1950s

After the end of World War II, all the Western allies began a rapid drawdown. At the end of WWII, the US still had a COMINT organization split between the Army and Navy. ^[23] A 1946 plan listed Russia, China, and a [redacted] country as high-priority targets.

Post Cold War

1980s US Tactical SIGINT policy and doctrine

After the Beirut deployment, the US Marine Corps did an after-action review of the 2nd Radio Battalion detachment that went with that force. Lieutenant General Alfred M. Gray, Jr. then commanding Fleet Marine Force Atlantic, and Lieutenant Colonel Bill Keller, commanding 2nd Radio Battalion, did an after-action review. Part of the reason for this was that the irregular units that presented the greatest threat did not follow conventional military signal operating procedures, and used nonstandard frequencies and callsigns. Without NSA information on these groups, the detachment had to acquire this information from their own resources.

Recognizing that national sources simply might not have information on a given environment, or that they might not make it available to warfighters, Lieutenant General Gray directed that a SIGINT function be created that could work with the elite Force Reconnaissance Marines who search out potential enemies. At first, neither the Force Reconnaissance nor Radio Battalion commanders though this was viable, but had orders to follow.

Initially, they attached a single Radio Battalion Marine, with an AN/GRR-8 intercept receiver, to a Force Reconnaissance team during an exercise. A respected Radio Marine, Corporal Kyle O'Malley was sent to the team, without any guidance for what he was to do. The exercise did not demonstrate that a one-man attachment, not Force Recon qualified, was useful.

In 1984, Captain E.L. Gillespie, assigned to the Joint Special Operations Command, was alerted that he was to report to 2nd Radio Battalion, to develop a concept of operations for integrating SIGINT capabilities with Force Recon, using his joint service experience with special operations. Again, the immediate commanders were not enthusiastic.

Nevertheless, a mission statement was drafted: "To conduct limited communications intelligence and specified electronic warfare operations in support of Force Reconnaissance operations during advance force or special operations missions." It was decided that a 6-man SIGINT team, with long/short range independent communications and SIGINT/EW equipment, was the minimum practical unit. It was not practical to attach this to the smallest 4-man Force Recon team.

General Gray directed that the unit would be called a Radio Reconnaissance Team (RRT), and that adequate planning and preparation were done for the advance force operations part of the upcoming Exercise Solid Shield-85. Two six-man teams would be formed, from Marines assigned from the Radio Battalion, without great enthusiasm for the assignment. One Marine put it"There is nothing that the Marine Corps can do to me that I can't take." ^[24] Force Recon required that the RRT candidates pass their selection course, and, to the surprise of Force Recon, they passed with honors. Both teams were assigned to the exercise, and the RRTs successfully maintained communications connectivity for Force Recon and SEALs, collected meaningful intelligence, disrupted opposing force communications, and were extracted without being compromised.

From 1986 on, RRTs accompanied MEU (SOC) deployments. Their first combat role was in Operation Earnest Will, then Operation Praying Mantis, followed by participation in the 1989 United States invasion of Panama

1990s

Terrorism from foreign groups became an increasingly major concern, as with the 1992 al-Qaeda attack in Yemen, the 1993 truck bombing of the World Trade Center, 1995 (Saudi communications center) and 1996 (Khobar Towers) in Saudi Arabia, and the 1998 bombings of the US embassies in Dar es Salaam, Tanzania and Nairobi, Kenya. Third world and non-national groups, with modern communications technology, in many ways are a harder SIGINT target than a nation, such as Russia or China, that sends out large amounts of traffic. According to the retired Commandant of the US Marines, Alfred M. Gray, Jr., some of the significant concerns of these targets are:

• Inherently low probability of intercept/detection (LPI/LPD) because off-the-shelf radios can be frequency agile, spread spectrum, and transmit in bursts.
• Additional frequencies, not normally monitored, can be used. These include citizens band, marine (MF, HF, VHF) bands, and higher frequencies for short-range communications
• Extensive use of telephones, almost always digital. Cellular and satellite telephones, while wireless, are challenging to intercept, as is Voice over IP (VoIP)
• Commercial strong encryption for voice and data
• "Extremely wide variety and complexity of potential targets, creating a "needle in the haystack" problem"^[25]

France

While the Helios satellite was IMINT, not SIGINT, it helped put perspective on program costs. [Helios 1A was launched on 7 July 1995.^[26] The Cerise (satellite) SIGINT technology demonstrator also was launched in 1995; it is not clear if it was on the Helios 1 launch. A radio propagation experiment, S80-T, was launched in 1992, as a predecessor of the ELINT experiments.

Financial pressures in 1994-1995 caused France to seek Spanish and Italian cooperation for Helios 1 and German contributions to HELIOS 2.^[27] Helios 2A was launched in 2004. France, still desiring to have three different space-based intelligence systems (IMINT, radar surveillance, SIGINT), had to face extremely high costs. In 1994-1995, French legislators tried to reduce some of these plans. In response, the French government sought Italian and Spanish funding in, and cooperation with, the HELIOS 1 program. They also sought German involvement in Helios 2. The HELIOS 2A launch also was accompanied by a small constellation of ELINT satellites.

The Cerise satellite ELINT technology demonstrator, also launched in 1995, was damaged by a collision with another French payload, SPOT-1, in the following year.

Clementine, the second-generation ELINT technology demonstrator, was launched in 1999.

United Kingdom

Controversy arose over alleged British interception of communications to Ireland from a facility called the Ministry of Defence Electronic Test Facility in a British Nuclear Fuels Limited site at Capenhurst, Cheshire. This facility was in the line of microwave towers from the UK-Ireland 1 cable (Dublin to Anglesey) landing to BT in London. Besides the Capenhurst tower, communications to and from the Irish Republic were also intercepted at a similar but A smaller GCHQ station in County Armagh was said to target links between Dublin and Belfast, and a third station intercepted satellite communications in Cornwall.^[28] Irish politicians, led by former Prime Minister Albert Reynolds, demanded an investigation.

2000s

As evidenced by the Hainan Island incident, even while China and the US may cooperate on matters of mutual concern towards Russia, the Cold War has not completely disappeared.

There was more regional cooperation, often driven by concerns about transnational terrorism. European countries also are finding that by sharing the cost, they can acquire SIGINT, IMINT, and MASINT capabilities independent of the US.

In the US, both communications security and COMINT policies have been evolving, some with challenges. The adoption of a Belgian-developed encryption algorithm, approved in a public process, and accepted both for sensitive but unclassified traffic, as well as for classified information sent with NSA-generated and maintained keys, redraws the cryptologic environment as no longer NSA or not-NSA. Controversy continues on various types of COMINT justified as not requiring warrants, under the wartime authority of the President of the United States.

Technologically, there was much greater use of UAVs as SIGINT collection platforms.

Hainan Island incident

In 2001, a US EP-3 SIGINT aircraft collided with a shadowing Chinese fighter, in what became known as the Hainan Island incident. Each side blamed the other; the US claimed the aircraft was in international airspace. The fighter pilot died, and the EP-3 made an emergency landing in China, erasing as much sensitive information as possible. While the Chinese released the aircraft several months after releasing the crew, the most sensitive information was not so much the aircraft's instrumentation, but the signals it was targeting and the reference material about the Chinese "electronic order of battle".

European Space Systems cooperation

French initiatives, along with French and Russian satellite launching, have led to cooperative continental European arrangements for intelligence sensors in space. In contrast, the UK has reinforced cooperation under the UKUSA agreement.

French space-based intelligence

On 18 December 2004,^[29] HELIOS 2A, built by EADS-Astrium for the French Space Agency (CNES), was launched into a Sun-synchronous polar orbit at an altitude of about 680 kilometers. There it will serve the French defense ministry, as well as cooperating European countries. HELIOS 2B is scheduled for launch in 2008.

The same launcher carried French and Spanish scientific satellites and four Essaim ("Swarm") experimental ELINT satellites^[30] .^[31]

Sources in the French procurement agency, DGA, confirmed Essaim, a system of ground station and satellite constellation, is working well.^[32] There have been French defense complaints about Essaim being a third technology demonstrator, after the 1995 Cerise and 1999 Clementine. DGA countered that Essaim will demonstrate more advanced technology, important to convince other European governments to help with the cost. Essaim is to provide some operational data. The first of three ground stations is operational, with three satellites in operation and the fourth considered an in-orbit spare.

In a Ministère de la Défense 18 December 2004 statement, France announced^[33] that Helios 2A is part of an exchange program planned with the German SAR Lupe and Italian COSMO-SKYMED systems, under development respectively in Germany and Italy.

German Space Systems

Following the first successful launch on December 19, 2006, about a year after the intended launch date, further satellites were launched at roughly six-month intervals, and the entire system of this five-satellite SAR Lupe synthetic aperture radar constellation achieved full operational readiness on 22 July 2008.^[34]

SAR is usually considered a MASINT sensor, but the significance here is that Germany obtains access to French satellite ELINT.

Italian Space Systems

With the first satellite launched on June 8, 2007,^[35] Italy and France are cooperating on the deployment of the dual-use Orfeo civilian and military satellite system.^[36]

Orfeo is a dual-use (civilian and military) earth observation satellite network developed jointly between France and Italy. Italy is developing the Cosmo-Skymed X-band polarimetric SAR, to fly on two of the satellites. The other two will have complementary French electro-optical payloads. The second Orfeo is scheduled to launch in early 2008.

While this is not an explicit SIGINT system, the French-Italian cooperation may suggest that Italy can get data from the French Essaim ELINT microsatellites.

Acceptance of cryptologic expertise outside NSA

The US government withdrew the last approvals for the Data Encryption Standard, approved for unclassified use in 1976 but now considered quite vulnerable. Its replacement, the Advanced Encryption Standard (AES) was approved in 2002. AES, when used with NSA-supplied keys, is approved for TOP SECRET traffic as well as unclassified, and may be considered a reference point for strong commercial encryption. AES appears, at the present time, to be secure when used properly, which represents a major change in US policy about the availability of strong communications security. Not all governments will allow the use of such strong ciphers.

That the algorithm chosen came from Europe points to a more multilateral world with respect to communications security. AES was developed by two Belgian cryptographers, Joan Daemen and Vincent Rijmen, and submitted to the AES selection process under the name "Rijndael", a portmanteau of the names of the inventors.

Terrorism and response in the US

As a result of the 9/11 attacks, intensification of US intelligence efforts, domestic and foreign, were to be expected. A key question, of course, was whether US intelligence could have prevented or mitigated the attacks, and how it might prevent future attacks. There is a continuing clash between advocates for civil liberties and those who assert that their loss is an agreeable exchange for enhanced safety.

SIGINT and the 9/11 attacks

In a statement to a joint meeting of the House and Senate Intelligence Committees, NSA Director LTG Michael Hayden said "NSA had no SIGINT suggesting that al-Qa'ida was specifically targeting New York and Washington, D.C., or even that it was planning an attack on U.S. soil. Indeed, NSA had no knowledge before September 11 that any of the attackers were in the United States, although the FBI was tailing them. See Able Danger.

"We are digging out of a deep hole. NSA downsized about one-third of its manpower and about the same proportion of its budget in the decade of the 1990s. That is the same decade when packetized communications (the e-communications we have all become familiar with) surpassed traditional communications. That is the same decade when mobile cell phones increased from 16 million to 741 million an increase of nearly 50 times. That is the same decade when Internet users went from about 4 million to 361 million an increase of over 90 times. Half as many landlines were laid in the last six years of the 1990s as in the whole previous history of the world. In that same decade of the 1990s, international telephone traffic went from 38 billion minutes to over 100 billion. This year, the world's population will spend over 180 billion minutes on the phone in international calls alone.

"throughout the summer of 2001 we had more than 30 warnings that something was imminent. We dutifully reported these, yet none of these subsequently correlated with terrorist attacks. The concept of "imminent" to our adversaries is relative; it can mean soon or imply sometime in the future"^[37]

US domestic surveillance issues

Under the George W. Bush administration, there has been a large-scale and controversial capture and analysis of domestic and international telephone calls, claimed to be targeted against terrorism. It is generally accepted that warrants have not been obtained for this activity, sometimes called Room 641A after a location, in San Francisco, where AT&T provides NSA access. While very little is known about this system, it may be focused more on the signaling channel and Call detail records than the actual content of conversations.

Another possibility is the use of software tools that do high-performance deep packet inspection. According to the marketing VP of Narus, "Narus has little control over how its products are used after they're sold. For example, although its lawful-intercept application has a sophisticated system for making sure the surveillance complies with the terms of a warrant, it's up to the operator whether to type those terms into the system...

"That legal eavesdropping application was launched in February 2005, well after whistle-blower Klein allegedly learned that AT&T was installing Narus boxes in secure, NSA-controlled rooms in switching centers around the country. But that doesn't mean the government couldn't write its own code to do the dirty work. Narus even offers software-development kits to customers ".^[38] The same type of tools with legitimate ISP security applications also have COMINT interception and analysis capability.

Former AT&T technician Mark Klein, who revealed AT&T was giving NSA access, said in a statement, said a Narus STA 6400 was in the NSA room to which AT&T allegedly copied traffic. The Narus device was "known to be used particularly by government intelligence agencies because of its ability to sift through large amounts of data looking for preprogrammed targets."^[38]

References

Bibliography

• Aldrich, R. J. (2010), GCHQ: The Uncensored Story of Britain's Most Secret Intelligence Agency, London: HarperCollins Press, ISBN 978-0-00-727847-3 
• Alvarez, David (2001), "Most Helpful and Cooperative: GC&CS and the Development of American Diplomatic Cryptanalysis, 1941-1942", in Smith, Michael; Erskine, Ralph, Action This Day: Bletchley Park from the Breaking of the Enigma Code to the Birth of the Modern Computer, Bantam Press, ISBN 978-0593049105 
• Budiansky, Stephen (2000), Battle of wits: The Complete Story of Codebreaking in World War II, Free Press, pp. 176–179, ISBN 978-0684859323 
• Holmes, W. J. (1979), Double-edged Secrets: U.S. Naval Intelligence Operations in the Pacific During World War II, Annapolis, Maryland: Blue Jacket Books/Naval Institute Press, ISBN 1-55750-324-9 
• Kahn, David (1996), The Codebreakers: The Comprehenisve History of Secret Communication from Ancient Times to the Internet, New York: Scribner, ISBN 0684831309 
• Lee, Bartholomew (2006), Radio Spies -- Episodes in the Ether Wars, retrieved 2007-10-08 
• Parker, Frederick D., "A Priceless Advantage: U.S. Navy Communications Intelligence and the Battles of Coral Sea, Midway, and the Aleutians", A Priceless Advantage: U.S. Navy Communications Intelligence and the Battles of Coral Sea, Midway, and the Aleutians (National Security Agency, Central Security Service), retrieved November 20, 2006 
• Prados, John (1995), Combined Fleet Decoded: The Secret History of American Intelligence and the Japanese Navy in World War II, New York: Random House, ISBN 0-679-43701-0 

Intelligence management Collection Interpersonal (HUMINT) Special reconnaissance (organizations) Clandestine Asset recruiting Cell system Covert action Direct action Operational techniques Espionage Agents (field handling) Asset Black operation (black bag) Concealment device Cryptography Cut-out Dead drop Denial and deception Eavesdropping False flag Industrial espionage Intelligence assessment Interrogation Numbers station One-way voice link Official cover Non-official cover Resident spy Steganography Surveillance Signals (SIGINT) By alliances, nations and industries In modern history Operational platforms by nation Direction finding Traffic analysis TEMPEST Measurement and signature (MASINT) Electro-optical Geophysical Nuclear Radar Radiofrequency Materials Casualty estimation (earthquake) Other Cultural (CULTINT) Financial (FININT) Geospatial (GEOINT) Imagery (IMINT) Market (MARKINT) Open-source (OSINT) Technical (TECHINT) Analysis Cognitive traps Competing hypotheses Target-centric Words of estimative probability Dissemination Intelligence cycle security Counterintelligence (organizations failures) Counterintelligence and counter-terrorism organizations