Venona project

The Venona project was a counter-intelligence program initiated by the United States Army Signal Intelligence Service (a forerunner of the National Security Agency) that lasted from 1943 to 1980.^[2] The program attempted to decrypt messages sent by Soviet Union intelligence agencies, including its foreign intelligence service and military intelligence services.^[3] During the program's four decades, approximately 3,000 messages were at least partially decrypted and translated.^[4] The project produced some of the most important breakthroughs for western counter-intelligence in this period, including the discovery of the Cambridge spy ring^[5] and the exposure of Soviet espionage targeting the Manhattan Project.^[6] The project was one of the most sensitive secrets of United States intelligence. It remained secret for over a decade after it ended and was not officially declassified until 1995.

Background

During the initial years of the Cold War, the Venona project was a source of information on Soviet intelligence-gathering activity that was directed at the Western military powers. Although unknown to the public, and even to Presidents Franklin D. Roosevelt and Harry S. Truman, these programs were of importance concerning crucial events of the early Cold War. These included the Julius and Ethel Rosenberg spying case and the defections of Donald Maclean and Guy Burgess to the Soviet Union.

Most decipherable messages were transmitted and intercepted between 1942 and 1945. Sometime in 1945, the existence of the Venona program was revealed to the Soviet Union by the NKVD agent and United States Army SIGINT analyst and cryptologist Bill Weisband.^[7] These messages were slowly and gradually decrypted beginning in 1946 and continuing (many times at a low-level of effort in the latter years) through 1980, when the Venona program was terminated, and the remaining amount of effort that was being spent on it was moved to more important projects.

To what extent the various individuals were involved with Soviet intelligence is a topic of dispute. While a number of academics and historians assert that most of the individuals mentioned in the Venona decrypts were most likely either clandestine assets and/or contacts of Soviet intelligence agents,^[8][9] others argue that many of those people probably had no malicious intentions and committed no crimes.^[10][11][12]

Commencement

The Venona Project was initiated in 1943, under orders from the deputy Chief of Military Intelligence (G-2), Carter W. Clarke.^[13] Clarke distrusted Joseph Stalin, and feared that the Soviet Union would sign a separate peace with the Third Reich, allowing Germany to focus its military forces against Great Britain and the United States.^[14] Code-breakers of the US Army's Signal Intelligence Service (commonly called Arlington Hall) analyzed encrypted high-level Soviet diplomatic intelligence messages intercepted in large volumes during and immediately after World War II by American, British, and Australian listening posts.^[15]

Decryption

This message traffic, which was encrypted with a one-time pad system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early 1940s. Due to a serious blunder on the part of the Soviets, some of this traffic was vulnerable to cryptanalysis. Somebody who was working for the manufacturers of Soviet secret-communication materials had reused pages of some of the one-time pads in other pads, which were then used for other secret messages. This defeated the purpose of the one-time pad, which provides perfect security when each page is used exactly once and then disposed of. It is unclear as to why this mistake was made, or by whom.

Breakthrough

The Soviet systems in general used a code to convert words and letters into numbers, to which additive keys (from one-time pads) were added, encrypting the content. When used correctly, one-time pad encryption is unbreakable.^[16] Cryptanalysis by American and British code-breakers revealed that some of the one-time pad material had incorrectly been reused by the Soviets (specifically, entire pages, although not complete books), which allowed decryption (sometimes only partial) of a small part of the traffic.

Generating the one-time pads is a slow and labor-intensive process, and the outbreak of war with Germany in June 1941 caused a sudden increase in the need for coded messages. It is probable that the Soviet code generators started duplicating cipher pages in order to keep up with demand.

It was Arlington Hall's Lieutenant Richard Hallock, working on Soviet "Trade" traffic (so called because these messages dealt with Soviet trade issues), who first discovered that the Soviets were reusing pages. Hallock and his colleagues (including Genevieve Feinstein, Cecil Phillips, Frank Lewis, Frank Wanat, and Lucille Campbell) went on to break into a significant amount of Trade traffic, recovering many one-time pad additive key tables in the process.

A young Meredith Gardner then used this material to break into what turned out to be NKVD (and later GRU) traffic by reconstructing the code used to convert text to numbers. Samuel Chew and Cecil Phillips also made valuable contributions. On 20 December 1946, Gardner made the first break into the code, revealing the existence of Soviet espionage in the Manhattan Project.^[17] Venona messages also indicated that Soviet spies worked in Washington in the State Department, Treasury, Office of Strategic Services, and even the White House. Very slowly, using assorted techniques ranging from traffic analysis to defector information, more of the messages were decrypted.

Claims have been made that information from the physical recovery of code books (a partially burned one was obtained by the Finns) to bugging embassy rooms in which text was entered into encrypting devices (analyzing the keystrokes by listening to them being punched in) contributed to recovering much of the plaintext. These latter claims are less than fully supported in the open literature.

One significant aid (mentioned by the NSA) in the early stages may have been work done in cooperation between the Japanese and Finnish cryptanalysis organizations; when the Americans broke into Japanese codes during World War II, they gained access to this information. There are also reports that copies of signals purloined from Soviet offices by the Federal Bureau of Investigation (FBI) were helpful in the cryptanalysis. The Finnish radio intelligence sold much of its material concerning Soviet codes to OSS in 1944 during Operation Stella Polaris, including the partially burned code book.^{[citation needed]}

Results

NSA reported, according to the serial numbers of the Venona cables, thousands were sent, but only a fraction were available to the cryptanalysts. Approximately 2,200 messages were decrypted and translated; about half for the 1943 GRU-Naval Washington to Moscow messages were broken, but none for any other year, although several thousand were sent between 1941 and 1945. The decryption rate of the NKVD cables was as follows:

• 1942 1.8%
• 1943 15.0%
• 1944 49.0%
• 1945 1.5%

Out of some hundreds of thousands of intercepted encrypted texts, it is claimed under 3,000 have been partially or wholly decrypted. All the duplicate one-time pad pages were produced in 1942, and almost all of them had been used by the end of 1945, with a few being used as late as 1948. After this, Soviet message traffic reverted to being completely unreadable.^[18]

The existence of Venona decryption became known to the Soviets within a few years of the first breaks.^{[citation needed]} It is not clear whether the Soviets knew how much of the message traffic or which messages had been successfully decrypted. At least one Soviet penetration agent, British Secret Intelligence Service representative to the U.S., Kim Philby, was told about the project in 1949, as part of his job as liaison between British and U.S. intelligence. Since all of the duplicate one-time pad pages had been used by this time, the Soviets apparently did not make any changes to their cryptographic procedures after they learned of Venona. However, this information allowed them to alert those of their agents who might be at risk of exposure due to the decryption.

Significance

The decrypted messages gave important insights into Soviet behavior in the period during which duplicate one-time pads were used. With the first break into the code, Venona revealed the existence of Soviet espionage^[19] at Los Alamos National Laboratories.^[20] Identities soon emerged of American, Canadian, Australian, and British spies in service to the Soviet government, including Klaus Fuchs, Alan Nunn May, and Donald Maclean. Others worked in Washington in the State Department, the Treasury, Office of Strategic Services,^[21] and even the White House.

The decrypts show the U.S. and other nations were targeted in major espionage campaigns by the Soviet Union as early as 1942. Among those identified are Julius and Ethel Rosenberg; Alger Hiss; Harry Dexter White,^[13] the second-highest official in the Treasury Department; Lauchlin Currie,^[22] a personal aide to Franklin Roosevelt; and Maurice Halperin,^[23] a section head in the Office of Strategic Services.

The identification of individuals mentioned in Venona transcripts is sometimes problematic, since people with a "covert relationship" with Soviet intelligence are referenced by cryptonyms.^[24] Further complicating matters is the fact the same person sometimes had different cryptonyms at different times, and the same cryptonym was sometimes reused for different individuals. In some cases, notably Hiss, the matching of a Venona cryptonym to an individual is disputed. In many other cases, a Venona cryptonym has not yet been linked to any person. According to authors John Earl Haynes and Harvey Klehr, the Venona transcripts identify approximately 349 Americans whom they claim had a covert relationship with Soviet intelligence, though fewer than half of these have been matched to real-name identities.^[25] However, not every agent may have been communicating directly with Soviet intelligence. Each of those 349 persons may have had many others working for, and reporting only to, them.

The Office of Strategic Services, the predecessor to the CIA, housed at one time or another between fifteen and twenty Soviet spies.^[26] Duncan Lee, Donald Wheeler, Jane Foster Zlatowski, and Maurice Halperin passed information to Moscow. The War Production Board, the Board of Economic Warfare, the Office of the Coordinator of Inter-American Affairs and the Office of War Information, included at least half a dozen Soviet sources each among their employees. In the opinion of some, almost every American military and diplomatic agency of any importance was compromised to some extent by Soviet espionage.^[27]

Some scholars and journalists dispute the claims by Haynes, Klehr, and others concerning the precision of the matching of cryptonyms to actual persons.^[28] Also contested is the implication that all 349 persons identified had an intentional "covert relationship" with Soviet intelligence; it is argued, in some cases, the individual may have been an unwitting information source or a prospect for future recruitment by Soviet intelligence.

Bearing of Venona on particular cases

Venona has added information—some unequivocal, some ambiguous—to several espionage cases. Some known spies, including Theodore Hall, were neither prosecuted nor publicly implicated, because the Venona evidence against them was not made public.

The identity of Soviet source cryptonymed '19' remains unclear. According to British writer Nigel West, '19' was president of Czechoslovak government-in-exile Edvard Beneš.^[29] Military historian Eduard Mark^[30] and American authors Herbert Romerstein and Eric Breindel concluded it was Roosevelt's aide Harry Hopkins.^[31] According to American authors John Earl Haynes and Harvey Klehr, source code-named '19' could be someone from the British delegation to the Washington Conference in May 1943.^[32] Moreover, they argue no evidence of Hopkins as an agent has been found in other archives, and the partial message relating to "19" does not indicate if this source was a spy.^[33]

However, Vasily Mitrokhin was a KGB archivist who defected from the Soviet Union with copies of KGB files. He claimed Harry Hopkins was a secret Russian agent.^[34] Moreover, Oleg Gordievsky, a high-level KGB officer who also defected from the Soviet Union, reported that Iskhak Ahkmerov, the KGB officer who controlled the clandestine Soviet agents in the U.S. during the war, had said Hopkins was “the most important of all Soviet wartime agents in the United States.”^[35] In addition, Major George Racey Jordon stated he strongly suspected Hopkins was a spy based on activities he observed in his support role in the project known as "Lend-Lease."^[36]

Julius and Ethel Rosenberg

Venona has added significant information to the case of Julius and Ethel Rosenberg, making it clear Julius was guilty of espionage, but also showing that Ethel was probably no more than an accomplice, if that.^{[citation needed]} Venona and other recent information has shown, while the content of Julius' atomic espionage was not as vital as alleged at the time of his espionage activities, in other fields it was extensive. The information Rosenberg passed to the Soviets concerned the proximity fuze, design and production information on the Lockheed P-80 jet fighter, and thousands of classified reports from Emerson Radio. The Venona evidence indicates unidentified sources code-named "Quantum" and "Pers" who facilitated transfer of nuclear weapons technology to the Soviet Union from positions within the Manhattan Project. According to Alexander Vassiliev's notes from KGB archive, "Quantum" was Boris Podolsky and "Pers" was Russell W. McNutt, an engineer from the uranium processing plant in Oak Ridge.^[37][38]

Alger Hiss and Harry Dexter White

According to the Moynihan Commission on Government Secrecy, the complicity of both Alger Hiss and Harry Dexter White is conclusively proven by Venona,^[39][40] stating "The complicity of Alger Hiss of the State Department seems settled. As does that of Harry Dexter White of the Treasury Department." In his 1998 book, Senator Moynihan expresses certainty about Hiss's identification by Venona as a Soviet spy, writing "Hiss was indeed a Soviet agent and appears to have been regarded by Moscow as its most important."^[41]

Several current authors, researchers, and archivists consider the Venona evidence on Hiss to be inconclusive^[42] or incorrect.^[43]

Donald Maclean and Guy Burgess

When Kim Philby learned of Venona in 1949, he obtained advance warning that his fellow Soviet spy Donald Maclean was in danger of being exposed. The FBI told Philby about an agent cryptonymed 'Homer', whose 1945 message to Moscow had been decoded. As it had been sent from New York and had its origins in the British Embassy in Washington, Philby, who would not have known Maclean's cryptonym, deduced the sender's identity. By early 1951, Philby knew U.S. intelligence would soon also conclude Maclean was the sender, and advised Moscow to recall Maclean. This led to Maclean and Guy Burgess' flight to Russia in May 1951.^[44]

Soviet espionage in Australia

In addition to British and American operatives, Australians collected Venona intercepts at a remote base in the Outback.^{[citation needed]} The Soviets remained unaware of this base as late as 1950.^[45]

The founding of the Australian Security Intelligence Organisation by Labor Prime Minister Ben Chifley in 1949 was considered highly controversial within Chifley's own party.^{[citation needed]} Until then, the left-leaning Australian Labor Party had been hostile to domestic intelligence agencies on civil-liberties grounds and a Labor government founding one seemed a surprising about-face.^{[citation needed]} But the presentation of Venona material to Chifley, revealing evidence of Soviet agents operating in Australia, brought this about. As well as Australian diplomat suspects abroad, Venona had revealed Wally Clayton (cryptonym 'KLOD'), a leading official within the Communist Party of Australia (CPA), as the chief organiser of Soviet intelligence gathering in Australia.^[46] Investigation revealed that Clayton formed an underground network within the CPA so that the party could continue to operate if it were banned.^{[citation needed]}

Public disclosure

For much of its history, knowledge of Venona was restricted even from the highest levels of government. Senior army officers, in consultation with the FBI and CIA, made the decision to restrict knowledge of Venona within the government (even the CIA was not made an active partner until 1952). Army Chief of Staff Omar Bradley, concerned about the White House's history of leaking sensitive information, decided to deny President Truman direct knowledge of the project. The president received the substance of the material only through FBI, Justice Department, and CIA reports on counterintelligence and intelligence matters. He was not told the material came from decoded Soviet ciphers. To some degree this secrecy was counter-productive; Truman was distrustful of FBI head J. Edgar Hoover and suspected the reports were exaggerated for political purposes.

Some of the earliest detailed public knowledge that Soviet code messages from World War II had been broken came with the release of Robert Lamphere's book, The FBI-KGB War, in 1986. Lamphere had been the FBI liaison to the code-breaking activity, had considerable knowledge of Venona and the counter-intelligence work that resulted from it. MI5 assistant director Peter Wright's 1987 memoir, Spycatcher, however, was the first detailed account of the Venona project, identifying it by name and making clear its long-term implications in post-war espionage.

Many inside the NSA had argued internally that the time had come to publicly release the details of the Venona project, but it was not until 1995 that the bipartisan Commission on Government Secrecy, with Senator Moynihan as chairman, released Venona project materials. Moynihan wrote:

"[The] secrecy system has systematically denied American historians access to the records of American history. Of late we find ourselves relying on archives of the former Soviet Union in Moscow to resolve questions of what was going on in Washington at mid-century. [...] the Venona intercepts contained overwhelming proof of the activities of Soviet spy networks in America, complete with names, dates, places, and deeds."^[47]

One of the considerations in releasing Venona translations was the privacy interests of the individuals mentioned, referenced, or identified in the translations. Some names were not released because to do so would constitute an invasion of privacy.^[48] However, in at least one case, independent researchers identified one of the subjects whose name had been obscured by the NSA.

The dearth of reliable information available to the public—or even to the President and Congress—may have helped to polarize debates of the 1950s over the extent and danger of Soviet espionage in the United States. Anti-Communists suspected many spies remained at large, perhaps including some known to the government. Those who criticized the governmental and non-governmental efforts to root out and expose communists felt these efforts were an overreaction (in addition to other reservations about McCarthyism). Public access—or broader governmental access—to the Venona evidence would certainly have affected this debate, as it is affecting the retrospective debate among historians and others now. As the Moynihan Commission wrote in its final report:

"A balanced history of this period is now beginning to appear; the Venona messages will surely supply a great cache of facts to bring the matter to some closure. But at the time, the American Government, much less the American public, was confronted with possibilities and charges, at once baffling and terrifying."

The National Cryptologic Museum features an exhibit on the Venona project in its "Cold War/Information Age" gallery.

Texas textbook controversy

Controversy arose in 2009 over the Texas State Board of Education's revision of their high school history class curricula to suggest Venona shows Senator Joseph McCarthy to have been justified in his zeal in exposing those whom he believed to be Soviet spies or communist sympathizers.^[49] Critics assert most people and organizations identified by McCarthy were not mentioned in the Venona content and that the sources for his accusations remain largely unknown.^[50] However, M. Stanton Evans in Blacklisted by History: The Untold Story of Senator Joe McCarthy and His Fight against America's Enemies reveals that his primary source was the files of the Federal Bureau of Investigation.^[51]

Critical views

A number of writers and scholars have taken a critical view of the translations released by the NSA. A few critics, including Robert and Michael Meeropol, the sons of Ethel and Julius Rosenberg, and activist lawyer William Kunstler, cast doubt on the authenticity of the Venona material, suggesting that it might have been at least in part forged.^[52]

Other critics have questioned the accuracy of the translations and the identifications of covernames that the NSA translations give. Writers Walter and Miriam Schneir, in a lengthy 1999 review of one of the first book-length studies of the messages, object to what they see as the book’s overconfidence in the translations’ accuracy, noting that the undecrypted gaps in the texts can make interpretation difficult, and emphasizing the problem of identifying the individuals mentioned under covernames.^[53] To support their critique, they cite a declassified memorandum, written in 1956 by A. H. Belmont, who was assistant to FBI director J. Edgar Hoover at the time.^[54] In the memo, Belmont discusses the possibility of using the VENONA translations in court to prosecute Soviet agents, and comes out strongly opposed to their use. His reasons include legal uncertainties about the admissibility of the translations as evidence, and the difficulties that prosecution would face in supporting the validity of the translations. Belmont highlights the uncertainties in the translation process, noting that the cryptographers have indicated that “almost anything included in a translation of one of these deciphered messages may in the future be radically revised.” He also notes the complexities of identifying people with covernames, describing how the personal details mentioned for covername “Antenna” fit more than one person, and the investigative process required to finally connect “Antenna” to Julius Rosenberg. The Schneirs conclude that "A reader faced with Venona's incomplete, disjointed messages can easily arrive at a badly skewed impression."^[55]

Many of the critiques of the Venona translations have been based on specific cases. The Schneirs' critique of the Venona documents was based on their decades of work on the case of Ethel and Julius Rosenberg. Another critique of the Venona translations came from the late Rutgers University law professor John Lowenthal, who as a law student worked as a volunteer for Alger Hiss's defense team, and later wrote extensively on the Hiss case.^[56] Lowenthal's critique focused on one message (Venona 1822 KGB Washington-Moscow 30 March 1945),^[57] in which the comments identified the covername 'Ales' as "probably Alger Hiss." Lowenthal raised a number of objections to this identification, rejecting it as "a conclusion psychologically motivated and politically correct but factually wrong."^[58] Lowenthal's article led to an extended debates on the 'Ales' message,^[59] and even prompted the NSA to declassify the original Russian text.^[60] Currently Venona 1822 is the only message for which the complete decrypted Russian text has been published.

Victor Navasky, editor and publisher of The Nation, has also written several editorials highly critical of John Earl Haynes' and Harvey Klehr's interpretation of recent work on the subject of Soviet espionage.^[28] Navasky claims the Venona material is being used to “distort … our understanding of the cold war” and that the files are potential “time bombs of misinformation.”^[10] Commenting on the list of 349 Americans identified by Venona, published in an appendix to Venona: Decoding Soviet Espionage in America, Navasky wrote, "The reader is left with the implication— unfair and unproven— that every name on the list was involved in espionage, and as a result, otherwise careful historians and mainstream journalists now routinely refer to Venona as proof that many hundreds of Americans were part of the red spy network."^[10] Navasky goes further in his defense of the listed people and has claimed a great deal of the so-called espionage that went on was nothing more than “exchanges of information among people of good will” and that “most of these exchanges were innocent and were within the law.”^[11]

According to historian Ellen Schrecker, "Because they offer insights into the world of the secret police on both sides of the Iron Curtain, it is tempting to treat the FBI and Venona materials less critically than documents from more accessible sources. But there are too many gaps in the record to use these materials with complete confidence."^[61]

Schrecker believes the documents established the guilt of many prominent figures, but is still critical of the views of scholars such as John Earl Haynes, arguing, "complexity, nuance, and a willingness to see the world in other than black and white seem alien to Haynes' view of history."^[62]

Despite the objections raised, however, the majority of historians remain convinced of the historical value of the Venona material. Intelligence historian Nigel West believes that "Venona remain[s] an irrefutable resource, far more reliable than the mercurial recollections of KGB defectors and the dubious conclusions drawn by paranoid analysts mesmerized by Machiavellian plots."^[63]

See also

• Espionage Act of 1917
• History of Soviet and Russian espionage in the United States
• List of Americans in the Venona papers
• List of Soviet agents in the United States
• McCarthyism
• Taman Shud Case

Notes

References and further reading

v t e National Security Agency Locations AMOC Ft. Meade Friendship Annex Georgia Cryptologic Center KRSOC MRSOC MSOC RAF Menwith Hill Room 641A Rosman Research Station Sugar Grove Yakima Training Center Leaders Ralph Canine John A. Samford Laurence Hugh Frost Gordon Blake Marshall Carter Noel Gayler Samuel C. Phillips Lew Allen Bobby Ray Inman Lincoln D. Faurer William Eldridge Odom Bill Studeman John Michael McConnell Kenneth Minihan Michael Hayden Keith B. Alexander Prominent figures Bernard Ardisana Howard C. Barlow William Blankinship Dorothy T. Blum Benson K. Buffman Lambros D. Callimahos James R. Chiles Mahlon E. Doyle Agnes Meyer Driscoll Thomas H. Dyer Edward A. Everett Lowell K. Frazer Elizebeth S. Friedman William F. Friedman Charles L. Gandy Meredith Gardner Alfred M. Gray Jacob Gurin Robert J. Hermann Sydney Jaffe David Kahn Oliver R. Kirby Richard A. Leibler Solomon Kullback Mitford M. Mathews Juanita Moody John E. Morrison Cecil J. Phillips James W. Pryde Francis Raven Joseph J. Rochefort Howard E. Rosenblum Frank Rowlett Laurance Safford Arthur Salemme Donald M. Showers Abraham Sinkov Samuel S. Snyder Charles C. Tevis John Tiltman Louis W. Tordella Thomas E. Tremain Julia Ward Joseph N. Wenger Norman Wild Herbert O. Yardley Milton Zaslow Operations Central Security Service (CSS) COMINT Information Assurance Remote Operations Center SIGINT Tailored Access Operations (TAO) NSA Threat Operations Center (NTOC) Technology ANT catalog FROSTBURG HARVEST Secure Terminal Equipment (STE) Secure Telephone Unit (STU)-1 STU-II STU-III WARRIOR PRIDE Controversy 2013 mass surveillance disclosures Church Committee Edward Snowden LOVEINT James Bamford NSA warrantless surveillance controversy Pike Committee Russ Tice Thomas Andrews Drake Thomas Tamm Programs Boundless Informant Dropmire ECHELON Fairview Insider Threat Program MUSCULAR PRISM Stellar Wind TRAILBLAZER Turbulence Upstream XKeyscore Databases Dishfire Main Core MAINWAY MARINA PINWALE Other Central Security Service Dundee Society National Cryptologic Museum National Cryptologic School National Vigilance Park NSA Hall of Honor Utah Data Center VENONA Zendian Problem