Klein worked for more than 20 years as a technician at AT&T. Here he tells the story of how he inadvertently discovered that the whole flow of Internet traffic in several AT&T operations centers was being regularly diverted to the National Security Agency (NSA). Klein is a witness in a lawsuit filed against AT&T by the Electronic Frontier Foundation (EFF), which alleges AT&T illegally gave the NSA access to its networks. This is the edited transcript of an interview conducted on Jan. 9, 2007.
What did you do for AT&T? How long did you work there?
I worked at AT&T for 22 and a half years. My job was basically to keep the systems going. They were computer systems, network communication systems, Internet equipment, Voice over Internet [Protocol (VoIP)] equipment. I tested circuits long distance across the country. That was my job: to keep the network up. ...
So you handled the hottest high-tech stuff that AT&T had.
That's right. Our job was to keep everything up and running smoothly.
What goes on inside the building on Folsom Street [in San Francisco]?
While I was there we worked on three floors which belonged to AT&T. The sixth floor was the traditional phone switch, ... which handled the public's telephone calls and was the workhorse of the phone system. The seventh floor was where the Internet room was, and that's where I spent a lot of my time. That's where there are a lot of Cisco routers, a lot of fiber-optic lines coming in and going out. The eighth floor is more diversified, more routers and other kinds of equipment, what's called multiplex equipment and various kinds of telecommunications equipment.
So this is an operations center. Are we talking about serving San Francisco? Are we talking about serving the state of California? Are we talking about America and Asia? What's the traffic that's going through there?
Well, this is an important hub for the Bay Area in terms of if you're talking about Internet. There's lots of Internet traffic, as you can imagine, that goes in and out of this office, probably hundreds of fiber-optic lines that go out, carrying billions -- that's billions with a "B" -- billions of bits of data going in and out every second every day. So all the Web surfing you're doing, whatever you're doing on the Internet -- the pictures, the video, the Voice over Internet -- all that stuff's going in and out of there.
And then of course there's also the traditional phone switch, which is doing what it's been doing since before the Internet.
Handling millions of calls. ...
Handling millions and millions of phone calls, right. That's its job.
So this is a big hub.
It's a big hub, yes.
... In 2002 I was sitting at my workstation one day, and some e-mail came in. I opened it up, and it was just a notice saying that somebody from the National Security Agency, NSA, was going to come visit for some business. They didn't say what, of course, just letting us know. I was also told the same thing by the manager of the office.
Don Henry, who mentioned that there was going to be a visit from this person from the National Security Agency. ... That struck me as a little odd to begin with, because I remember from back in the '70s, the NSA is not supposed to be doing domestic spying, so what were they doing in an AT&T company office? It struck me as odd, but I didn't know anything more about it, so I just let it lie and waited for the guy to come.
Sometime later, maybe a few weeks -- I don't remember exactly -- he did show up. This NSA representative showed up at the door. I happened to be the one who opened the door. I let him in. I directed him to the appropriate people. He was very closemouthed and unsmiling, and he did his business. ... I didn't hear anything about it for a while, and I thought, well, that was over and I'll probably never hear anything about it again. So I never kept the e-mail. I thought it was just routine e-mail, and I'd never hear about it again. That whole incident probably took place in, I think it was the summer of 2002.
Then in the fall we started hearing these stories. ... We'd start talking to those guys over there at Folsom Street, and we'd hear stories coming back, because the field support guys, they would work at several offices. The guy that the NSA had interviewed [at our office on Geary Street] we heard was working on something over at Folsom Street. And I heard from our manager, Don, that he's working on some new room that's being built -- not Don; I mean the other guy. And so people start speculating: Now, what's this new room being built?
At the same time I started getting more suspicious about it. This was around the end of 2002. You might recall there was a big blowup in the news about the Total Information Awareness [TIA] program, led by Adm. [John] Poindexter, which caused the big upsetness in Congress, because what Poindexter was proposing to do was draw in databases from everywhere -- and this was in The New York Times -- draw in Internet data, bank records, travel records, everything into one big conglomeration which could be searchable by the government so they could find out everything about what anybody's doing at any time of day. And all this would be done without any warrants. This is how it was presented by Poindexter himself in The New York Times, and that caused a great upset, brouhaha, in Congress. ...
You've seen these stories in the paper about Poindexter. So what's the scuttlebutt [in the office]? ...
We obviously started to speculate, because the stuff that was being talked about by Poindexter was he wanted to get, without warrants, huge databases so he could do searching and connect the dots, whatever that means, on everybody without warrants. ... And funny thing, by coincidence, here this NSA guy shows up in our office at the same time. And the Total Information Awareness Program is involved with the NSA and with DARPA, which is the Defense [Advanced Research] Projects Agency.
So I began to connect the two, because it seemed quite logical at least that if they are looking for large amounts of Internet data to sift through and vacuum up, what would be a perfect place? It would be in the Internet room at a place like AT&T. And lo and behold, the NSA guy shows up. ...
Then I started learning that it's not only a new room; it's a room that all the technicians cannot go into. Only the one guy -- a management guy, no union people -- a management field specialist with security clearance obviously given to him by the NSA, only he could go into this room, which was being built on the sixth floor, right next door to the phone switch room. So I got very worried about that. What does this mean? What are they doing there? ...
Then it started to get more interesting in January of 2003 when ... they brought us over there for a tour to see the office.
So in January 2003 you're invited over to this Folsom Street office for a tour, and this is the place where this room is being built.
We went for a tour -- it was me and three other technicians from the Geary Street office [who] went over for this tour of the Folsom Street office just so we could get to know the office and the people, because they were obviously eventually planning to bring us over there. ... So we did a tour of the office, and one of the technicians was walking around with us, and then we got to this secret room, and they were just finishing the outer door. Some workman was working on the lock, and I was standing there talking to him, and he suddenly talked [in a] very low voice like he didn't want to be overheard. He felt like this was something secret, you know, and he didn't know much about it, and he was saying: "None of us can go in there. It's all secret." This was not only an affront to the technicians; it was a violation of union rules, because they were obviously planning to install telecommunications equipment, which is supposed to be the jurisdiction of the union technicians. We had a contract. So the technicians were not only angry about this secret thing that they're not let in on, but also the fact that there's work there that they're excluded from. And they were told nothing about it. So that was it.
I got to see the room. It was right next door to the phone switch room. But I didn't know whether there was a definite connection between the two or not, or whether that was just a coincidence. At any rate, I thought the whole thing rather weird and sort of heightened my worry about this whole operation, that the technicians were excluded and that the NSA was involved. I thought this was real fishy. ...
So do you do anything about this? Do you talk to anybody about this? Do you write things down? What do you do with this worry of yours?
I thought about what to do about this, and I was basically scared for several reasons, one being, well, this is obviously secret. This is obviously some federal government secret operation that they don't want nosy people nosing around in, and if I started asking questions I could get into trouble. Furthermore, our jobs were in jeopardy anyway, because [we] were always getting wind that they were planning to close our previous office at Geary Street, and I didn't need to give them an excuse to fire me. So I thought after thinking about it that the best thing to do is not to say anything and just watch it.
So what's the next level of awareness? Do you get moved to Folsom Street? Is that when you learned something more? What happens?
The next level of awareness is when we are actually transferred to Folsom Street. Now, that didn't come until October 2003. ...
What's your job at Folsom Street when you get there, and does that give any more light on what's going on?
Yes. That's when I finally found out what they're doing, by sheer accident. ... My main assignment was to oversee the Internet room, and that meant keeping it going. If there were any trouble calls, I had to answer them. If there's any upgrading work to do, I had to either do it or arrange for others to do it in off hours. Just oversee the flow of work in the Internet room and watch things.
As it turned out, one of the key pieces of spy equipment they installed was in the Internet room, and I discovered that in the process of learning the job. When I got there in October there was still a guy there who had been there years and years and years. He was thoroughly familiar with the whole office, ... and he was showing me the ropes for the Internet room because he was, as it turned out, planning to retire, and he left a couple months later. But in those two months he was showing me the ropes, and at one point I asked him about this secret room, because the secret room is on the sixth floor; the Internet room is on the seventh floor. It didn't seem to be an obvious connection, and I had said to him, "Well, it seems to me that the secret room is right next to the phone switch room, so I assume they're listening to phone calls," and his answer was: "No. Internet." That was his instant answer. He said, "I'll show you."
He was referring to what I found later, was what we called the splitter cabinet. The seventh-floor Internet room has whole lineups of equipment, row after row after row of equipment. In one row, they installed a cabinet that had optical splitters in it. So there were optical splitters, which basically were connected by fiber-optic cable down to the secret room on the sixth floor. ...
What's this splitter doing?
The analogy I can give you, which most people are familiar with is, say you get cable TV in your living room and then want to watch all the channels you get in the living room, you want to get all those same channels in your bedroom. So they install on the cable what they call a splitter, which splits off all the signals, duplicates of the same signals which go to the bedroom. ...
What the splitter does is make a duplicate copy of all the signals going across the fiber-optic cables. ... We're talking about billions and billions of bits of data going across every second, right? And it's going into the router, and it's coming back from the routers in that office. So what they do with the splitter is they intercept that data stream and make copies of all the data, and those copies go down on the cable to the secret room. ...
How do you know for sure that the traffic that's being diverted by the splitters is actually going down to that secret room and not somewhere else in the AT&T system?
Well, several ways. First of all, I had the comment from the technician who worked there, who knows that office like the back of his hand. I also eventually got hold of the documents. The documents, what they are, are wiring documents, which show where things are connected to. They have locations on them, and the documents clearly show the splitter cabinet, which is in the seventh-floor Internet room, is connected down to the secret room on the sixth floor. ...
Did you ever go in the secret room and see the end of the traffic down at the sixth floor?
... There was one time; by accident I was in the secret room for about two minutes. The splitter cabinet was giving me problems because I had to connect circuits to it. Sometimes when you'd do that, the circuit wouldn't work, because the splitter would reduce the signal level. So I called in the field specialist who had access to the secret room, and I said to him: "I'm having a problem here. Can you help me with the splitter cabinet?" He looked in there, and he decided he needed another card to put in the splitter cabinet. So he went down to get it, and he invited me along to come. ... So I went down with him. He went up to the door, punched in the code, opened the door, invited me in. I looked around, and it looked like generally any other telecommunications site, with rows of cabinets and equipment in there. I didn't get, of course, to go look inside to see what the equipment was. ...
... Was there anything unusual about it?
Nothing unusual as far as -- if I didn't know that the NSA was involved, it would look like any other work space where telecom people work, with rows of cabinets with equipment inside them, humming.
Anybody working in there?
No, no. This one guy who had authorization to be in there would occasionally go work in there, but as far as I know, there was no one permanently stationed in there. ...
When you were in that room looking around, did you see anything that looked non-AT&T? Anything that looked NSA?
You had your suspicions, so you're going in that room with extra curiosity.
Yes, but that doesn't mean anything, and I wouldn't be surprised if there's no labels that say NSA in there. They were just normal labels on cabinets saying what the location was, just like any telecommunications equipment. I couldn't see inside the cabinets, but they looked from the sound and the vague, black shapes [as if] there were some kind of routers, which was what you'd expect in telecommunications equipment.
But the odd thing about the whole room, of course, was that I couldn't normally get in there, nor could any of the other union technicians. Only this one guy who had clearance from the NSA could get in there, so that changed the whole context of what this is about. ...
First of all, they wouldn't need the NSA for that purpose, and we would be allowed in by union contract to service the equipment as we always were. So anyway, there's that question right off the bat: Why NSA?
Now, in October, while I was working and learning the Internet room, I came across these three documents, which were documents that the technicians had that were given to the technicians so they would know how to install things like the splitter cabinet in particular, because it tells how things are wired up. Those documents were left lying around. Some of the technicians still had them. One of them was just left lying on top of a router. I picked it up, and I looked at it, and I brought them back to my desk, and when I started looking at it, I looked at it more, and I looked at it more, and finally it dawned on me sort of all at once, and I almost fell out of my chair, because this showed, first of all, what they had done, that they had taken working circuits, which had nothing to do with a splitter cabinet, and they had taken in particular what are called peering links which connect AT&T's network with the other networks. It's how you get the Internet, right? One network connects with another. So they took 16 high-speed peering links which go to places like Qwest [Communications] and Palo Alto Internet Exchange [PAIX] and places like that. … These circuits were working at one point, and the documents indicated in February 2003 they had cut into these circuits so that they could insert the splitter so that they can get the data flow from these circuits to go to the secret room.
So this data flow meant that they were getting not only AT&T customers' data flow; they were getting everybody else's data flow, whoever else might happen to be communicating into the AT&T network from other networks. So it was turning out to be like a large chunk of the network, of the Internet.
Did you see that in the documents? Did the documents show that in the designs?
You can see that in the document. ... It names the circuit IDs; it names the companies they belong to; and it shows the cut date. And they were all in February, when they were cut into the splitter.
February 2003. Then I was looking at the equipment list. All these three documents were obviously all part of the same project, which involved cutting this splitter cabinet in. I looked at the main one, which is called Study Group 3, San Francisco, kind of an innocent-sounding name. What are they studying?
On the equipment list were standard things ... like Juniper routers and Sun servers, which are very common, high-quality equipment, and Sun storage equipment to store data. And there was a whole list there.
But then there was one thing that was odd, because I didn't recognize it. It was not part of normal, day-to-day telecommunications equipment that I was familiar with in years of my work, and that was a Narus STA 6400. STA stands for Semantic Traffic Analyzer. I'd never heard of this, so I started doing a little Google research to find out what this is about; what's a Semantic Traffic Analyzer? And so I find, after doing some Googling, that it's not only designed for high-speed sifting through high-speed volumes of data, looking for something according to various program algorithms, something you'd think would be perfect for a spy agency.
Turns out it was perfect for a spy agency, and they were already using it and boasting about it. I found, for instance, there was a conference in 2003 in McLean, Va., whose agenda was posted on the Internet. I'm sure you know McLean, Va., is the hometown of the CIA. ... The sponsor of the show was Narus, and they posted the agenda for this computer show, which was semi-public, and everybody was there, from the phone companies like AT&T and Verizon to the intelligence agencies like the DEA [Drug Enforcement Administration] and the FBI and local police agencies and the FCC [Federal Communications Commission]. I have to assume the NSA was there, although they didn't list themselves. But one of the guys on the agenda was William Crowell. William Crowell was the former deputy director of the NSA. He was on one of the forum lists as a speaker, along with the founder of Narus and a whole bunch of them.
So when I saw all that, it all clicked together to me: "Oh, that's what they're doing. This is a spy apparatus. I'm not just imagining things." ...
When you spotted this, you'd been in the room; you've seen the splitter; you've now got the documents; you've seen the Narus; you've gone to the Internet; you've seen what this technology show is about. What is it you think is going on here? What's your reaction?
My reaction -- that's why I practically fell out of my chair -- was that from all the connections I saw, they were basically sweeping up, vacuum-cleaning the Internet through all the data, sweeping it all into this secret room. ... It's the sort of thing that very intrusive, repressive governments would do, finding out about everybody's personal data without a warrant. I knew right away that this was illegal and unconstitutional, and yet they were doing it.
So I was not only angry about it; I was also scared, because I knew this authorization came from very high up -- not only high up in AT&T, but high up in the government. So I was in a bit of a quandary as to what to do about it, but I thought this should be halted.
So what did you do?
Well, immediately I thought there's nothing safe that I could do, and I decided to do nothing but just write down what I know. ... By early January 2004 I thought I had a good sense that what they'd done was install a vacuum cleaner operation on the Internet, and I knew also that there was more than one room. I had that from several sources. First of all, the documents themselves make reference to other sites. At one point the document mentions a site in Atlanta. Common sense tells you if this is called Study Group 3, San Francisco, there's got to be at least a 1 and a 2, and probably a 4, 5, 6 and so forth.
Then when I was talking to a technician on the phone back East, and I was trying to troubleshoot a problem because I couldn't get the splitter to work right, this technician tells me, "Oh, yeah, we're having the same problem with the splitters going into other offices."
Did he say where?
My hair just stood on end when the person said that, and I said, "Other offices?" And she said, "Yeah, in Seattle and San Diego and Los Angeles and San Jose," all having the same problem. ... I started scribbling some of them down, because I knew this is important information. So then I knew right there, OK, this is a lot bigger. ...
So you've now got a picture that it's going on in ... the West Coast and Atlanta?
And then also I had another confirmation from the horse's mouth. The field support guy ... we knew him from various jobs, and he'd come in and shoot the breeze with us at the water fountain, so to speak -- and sometimes he got probably a little too talkative, and he would start talking about his secret work. And he pulled out this bunch of keys around his neck and said, "Oh, yeah, this one's for Seattle, and that one's for Los Angeles." He had a whole bunch of keys to service the other rooms on the West Coast. So that gave me a horse's-mouth confirmation that at least on the West Coast there's a bunch more rooms.
... As you get this picture of this spreading network, ... what do you think you're looking at?
I think I'm looking at something Orwellian. It's a government, many-tentacled operation to gather daily information on what everybody in the country is doing. Your daily transactions on the Internet can be monitored with this kind of system, not just your Web surfing. All kinds of business that people do on the Internet these days -- your bank transactions, your e-mail, everything -- it sort of opens a window into your entire private life, and that's why I thought of the term "Orwellian." As you know, in [George] Orwell's story , they have cameras in your house, watching you. Well, this is the next best thing. ...
... What was the problem you were having [with the splitter]?
The problem very simply is this: The signals that go across fiber optics are laser light signals. It's light basically that runs through a fiber optic, which is a clear glass fiber, and it has to be at a certain level for the routers to see the light and interpret the data correctly. If the light gets too low, just as if you get a weak flashlight with bad batteries, at a certain point it doesn't work. If the light level drops too low, the router starts dropping bits and getting errors, and eventually you get loss of signal, and it just doesn't work at all. ...
The effect of the splitter is to reduce the strength of the signal, and that may or may not cause a problem, depending on how much the signal is reduced.
... Would a telephone company normally put that kind of a splitter on its backbone Internet traffic and risk losing the strength of its signal?
Normally to my knowledge they would not, because, as you say, they would risk losing a signal, or the signal quality would drop. You want to have as few connections on your main data lines as possible, because each connection reduces the signal strength, and a splitter is a connection, and if you can avoid that, all the better. ...
Much later the president comes out and says, you know, we're just monitoring Al Qaeda communications with America, and we're following specific calls, and we're trying to track terrorists. How do you know that it isn't what they say it is; that the Narus or some other piece of equipment isn't just targeted in on 50 individuals in this area, and all those billions of pieces of data are just flowing off into the ether?
... The administration's first presentation of it is disingenuous. They present it as about phone calls. They're just watching a few bad people who make phone calls to Al Qaeda and the Middle East, and you notice they don't talk about the Internet hardly at all. That part of it hasn't been revealed, because if they did, Americans would realize it's not just a few people; it's everybody, because the data they're handing over is not selected out. When you run fiber optics through a splitter and you send all that data to a secret room, there's no selecting going on there at all. ...
And they could be getting domestic-to-domestic [communications]?
That's right. They have no way of sifting it out unless they look through it later. Now they can claim, "Oh, we are right as rain; we're only doing the legal thing and selecting out a few people that we're legally entitled to," but that's only after they get all the data. The analogy I use: If the government claims, "Well, when you do your taxes, why don't you just write me a blank check and we'll fill in the amount? Don't worry. We'll do it legal. We'll fill in the right amount," would you do that? Nobody would trust the government by writing a blank check to them. It's the same thing with the data we're giving them. ...
When the founders wrote the Fourth Amendment, they had a specific antagonism against what were called general warrants, as you might know. General warrants is when the British troops would come in with a warrant and say: "We have the right to search your house. We're looking for something. Looking for what? We can't tell you. We're going to ransack your house." That's a general warrant. They can turn your life upside down, and the colonialists [sic] hated that.
So the Fourth Amendment specifically bans general warrants. It calls for specific warrants in which the things to be seized and the persons to be seized are specifically named. There's a reason for that. It's to protect against arbitrary government power. And what they've done is to trample over the Fourth Amendment by basically instituting a general warrant on the Internet.
Why? Because I remember the last time this happened. ... I did my share of anti-war marches when that was an active thing back in the '60s, and I remember the violations and traffic transgressions that the government pulled back then for a war that turned out to be wrong, and a lot of innocent people got killed over it. And I'm seeing all this happening again, only worse. When the [NSA] got caught in the '70s doing domestic spying, it was a big scandal, and that's why Congress passed the FISA [Foreign Intelligence Surveillance Act] law, as you know, to supposedly take care of that. So I remember all that.
And the only way any law is worth anything is if there's a memory so that people can say: "Wait a minute. This happened before." And you've got to step forward and say: "I remember this. This is the same bad thing happening again, and there should be a halt to it." And I'm a little bit of that institutional memory in the country; that's all. ...
So you're writing notes down. You're writing up what you see.
In January 2004 I wrote down the basic outline of what I know. That was eventually posted on the Internet by somebody, not me. But that has the basic outline. And the end of May 2004 I retired. In September 2004 I wrote up some more notes to give a little idea how I came to know how the NSA was involved, because that wasn't in the earlier writeup that I did. But then I still sat on it. This was 2004; and then 2005 went by. The atmosphere was still kind of scary, and I just put it in a drawer and forgot about it.
And then came December 2005, and as you know, then The New York Times revealed that the government was doing illegal spying without a warrant. They seemed to be talking mainly about phone calls, but anyway, it was revealed that there was an illegal spying program going on, and I thought, ah, this would probably blow the whole thing, and I thought it would all come out, and I don't need to do anything.
Instead to my surprise, what came out was the government turned around and went on the offensive against anybody who would dare to criticize them. … They're issuing threats: Anyone who has a security clearance and spills any beans here is in for prosecution. That was deliberately said by them several times on TV to intimidate anybody in, say, the NSA who knew the truth, intimidate them so they would not come forward. So that silenced anybody in the intelligence community.
So you figured you're it.
So I figured I'm it, because see, I don't have a security clearance. They can't do anything to me about that. All I had and still have are some company documents and some knowledge of some illicit NSA installation at AT&T's network. And if anybody -- say, Congress -- was willing to follow the trail, I can give them all the names they want, and they can go up the hierarchy of AT&T all the way up to Dave Dorman, who was the president back then, and they can go even higher, and they can find out who is responsible for this, and they can ask them under oath and subpoena what the heck is going on here, if they had the will to do it. ...
... Do you decide to act? What do you do in early 2006?
Well, I decide to try to make it known to the media, try to line up help to make this known and try to get somebody to do something about it. So I went to civil liberties groups, gave them documents. I went to newspapers, the Los Angeles Times, later The New York Times. I sent letters out later to congressmen and senators. That turned out to be a dead end. I never got anything back from the Senate, and Congress was staying away from me like I was some kind of plague. No one would answer me -- to this day, none of them answered me. The L.A. Times eventually killed the story because, I was told, John Negroponte talked to their top guy.
The director of national intelligence.
That's right. The New York Times eventually did do a nice story about the initial documents and what they mean. They showed them to some experts. That was useful. But beyond that nothing much has happened.
The legal case I joined as a declaration -- I gave the Electronic Frontier Foundation [EFF] a declaration which basically laid out the facts as I know it. So I've become a witness in their case, and of course, as you know, that case is stalled because the government is still trying to get the whole thing dismissed on the grounds [that it would reveal] state secrets, so it's still in court. Nothing much has progressed.
Editor's Note: The case is currently on appeal before the 9th Circuit Court.
How do you feel now?
Well, I feel better that some of it has gotten out. When some of my documents were leaked out, they got all around the Internet, and I can see by reading the exchanges on the Internet that at least the tech community has got it and has an idea what the government is doing and doesn't like it.
I'd like to [go] further, though -- get the American people as a whole to understand what's going on, and that takes more effort by the media, which has been rather sparse in the coverage. The courts, of course, have been somewhat frozen by the government trying to block the case. And the Congress, again, is still frozen as far as I can tell. Even though it's changed hands and the Democrats control Congress, nobody's contacted me. So I am still frustrated by that side. ...
... There were terrorists who were living among us prior to 9/11. They were moving around; they were going to flight schools; they were renting apartments; they were traveling around. Doesn't the government need to do something in terms of gathering information to try to prevent the next terrorist attack?
I think if they needed anything, they had it already on the books. There's lots of -- maybe too much -- leeway for surveillance as it is. And they had lots of information that 9/11 was going to happen. But for some strange reason, they didn't act.
So I think you're asking this government -- which is full of prevarications and misleading statements and not very truthful and also a large component of simply incompetence -- handing them the keys to everybody's private information. I don't trust them with that. I think they're far more interested in just aggrandizing power for power's sake, and they're just using it as an excuse -- the so-called war on terror, which is their excuse for everything they do. Everything is aggrandizing power secretly, with no oversight. And I'm against that. It's dangerous. ...