N.S.A. Able to Foil Basic Safeguards of Privacy on Web

The National Security Agency is winning its long-running secret war on encryption, using supercomputers, technical trickery, court orders and behind-the-scenes persuasion to undermine the major tools protecting the privacy of everyday communications in the Internet age, according to newly disclosed documents.

This undated photo released by the United States government shows the National Security Agency campus in Fort Meade, Md.

This article has been reported in partnership among The New York Times, The Guardian and ProPublica based on documents obtained by The Guardian. For The Guardian: James Ball, Julian Borger, Glenn Greenwald. For The New York Times: Nicole Perlroth, Scott Shane. For ProPublica: Jeff Larson.

CITING EFFORTS TO EXPLOIT WEB James R. Clapper Jr., the director of national intelligence.

The agency has circumvented or cracked much of the encryption, or digital scrambling, that guards global commerce and banking systems, protects sensitive data like trade secrets and medical records, and automatically secures the e-mails, Web searches, Internet chats and phone calls of Americans and others around the world, the documents show.

Many users assume — or have been assured by Internet companies — that their data is safe from prying eyes, including those of the government, and the N.S.A. wants to keep it that way. The agency treats its recent successes in deciphering protected information as among its most closely guarded secrets, restricted to those cleared for a highly classified program code-named Bullrun, according to the documents, provided by Edward J. Snowden, the former N.S.A. contractor.

Beginning in 2000, as encryption tools were gradually blanketing the Web, the N.S.A. invested billions of dollars in a clandestine campaign to preserve its ability to eavesdrop. Having lost a public battle in the 1990s to insert its own “back door” in all encryption, it set out to accomplish the same goal by stealth.

The agency, according to the documents and interviews with industry officials, deployed custom-built, superfast computers to break codes, and began collaborating with technology companies in the United States and abroad to build entry points into their products. The documents do not identify which companies have participated.

The N.S.A. hacked into target computers to snare messages before they were encrypted. In some cases, companies say they were coerced by the government into handing over their master encryption keys or building in a back door. And the agency used its influence as the world’s most experienced code maker to covertly introduce weaknesses into the encryption standards followed by hardware and software developers around the world.

“For the past decade, N.S.A. has led an aggressive, multipronged effort to break widely used Internet encryption technologies,” said a 2010 memo describing a briefing about N.S.A. accomplishments for employees of its British counterpart, Government Communications Headquarters, or GCHQ. “Cryptanalytic capabilities are now coming online. Vast amounts of encrypted Internet data which have up till now been discarded are now exploitable.”

When the British analysts, who often work side by side with N.S.A. officers, were first told about the program, another memo said, “those not already briefed were gobsmacked!”

An intelligence budget document makes clear that the effort is still going strong. “We are investing in groundbreaking cryptanalytic capabilities to defeat adversarial cryptography and exploit Internet traffic,” the director of national intelligence, James R. Clapper Jr., wrote in his budget request for the current year.

In recent months, the documents disclosed by Mr. Snowden have described the N.S.A.’s reach in scooping up vast amounts of communications around the world. The encryption documents now show, in striking detail, how the agency works to ensure that it is actually able to read the information it collects.

The agency’s success in defeating many of the privacy protections offered by encryption does not change the rules that prohibit the deliberate targeting of Americans’ e-mails or phone calls without a warrant. But it shows that the agency, which was sharply rebuked by a federal judge in 2011 for violating the rules and misleading the Foreign Intelligence Surveillance Court, cannot necessarily be restrained by privacy technology. N.S.A. rules permit the agency to store any encrypted communication, domestic or foreign, for as long as the agency is trying to decrypt it or analyze its technical features.

The N.S.A., which has specialized in code-breaking since its creation in 1952, sees that task as essential to its mission. If it cannot decipher the messages of terrorists, foreign spies and other adversaries, the United States will be at serious risk, agency officials say.

Just in recent weeks, the Obama administration has called on the intelligence agencies for details of communications by leaders of Al Qaeda about a terrorist plot and of Syrian officials’ messages about the chemical weapons attack outside Damascus. If such communications can be hidden by unbreakable encryption, N.S.A. officials say, the agency cannot do its work.

John Markoff contributed reporting.

Senators Force Weaker Safeguards Against Cyberattacks

Strong opposition from Mr. McCain, Republican of Arizona, and others on behalf of the business community forced Democratic and Republican supporters of the legislation to drop provisions that would have given the federal government the power to enforce minimum standards on systems that run power plants, air traffic control systems, dams and similar facilities.

The Senate will debate the measure next week, even though the changes have raised new questions about its effectiveness.

“The key to successfully fighting this threat is not adding more bureaucrats or forcing industries to comply with government red tape,” Mr. McCain said Friday in a statement that announced that he and seven other Republican senators had introduced their own bill that calls for more information sharing among companies. “Instead, we must leverage the ingenuity and innovation of the private sector in partnership with the most effective elements of the federal government to address this emerging threat.”

Original versions of the bill, which was first drafted in 2009, called for giving the Department of Homeland Security the power to enforce minimum cybersecurity standards on infrastructure computer systems that, if damaged, would lead to mass casualties or economic loss.

But the U.S. Chamber of Commerce and other business lobbyists strongly objected, saying that such regulations would create a costly and cumbersome process.

The measure now before the Senate makes the minimum standards optional, dealing a significant setback to the administration, which had made legislation to safeguard computer systems a top national security priority this year. In April, the House passed its own version of the cybersecurity bill that encourages businesses and intelligence agencies to share information about attacks and threats to computer systems. Senate backers of the measure say their hope now is to pass the legislation and get into talks with the House. Even more attempts to change the Senate measure are expected.

James A. Lewis, a senior fellow at the Center for Strategic and International Studies and a cybersecurity expert, said the revised Senate measure did not provide any new powers to the federal government to protect computer systems of critical infrastructure.

“If it is passed, nobody will notice it,” he said. “You can do everything in the bill with an executive order.”

Dr. Lewis added: “The same way you wouldn’t say that we don’t need the F.A.A. because we can rely on incentives and a voluntary approach, we can’t rely on incentives and voluntary action for cybersecurity. Every day the risk gets bigger, it is not only countries but politically motivated individuals who can just download this stuff. A lot of us hoped Congress would have done better than this.”

Senator Joseph I. Lieberman, the independent from Connecticut who sponsored the measure as chairman of the Homeland Security and Governmental Affairs Committee, and the committee’s ranking member, Senator Susan Collins, Republican of Maine, said Friday that they were caught off guard by the determined opposition of Mr. McCain, who has for years made national security issues his priority.

“He knows that I’m disappointed,” Mr. Lieberman said, referring to Mr. McCain, one of his closest allies and friends in the Senate. “His natural side, based on his whole history, is to do the best thing for security and not to be worried about other factors.”

Tension between Mr. Lieberman and Mr. McCain bubbled to the surface on Wednesday at a closed-door meeting of senators and staff members about the legislation in the office of Senator Jon Kyl of Arizona, the No. 2 Senate Republican, said Congressional aides who attended the meeting and discussed the deliberations on the grounds that they would not be quoted by name.