Building the Kremlin’s Big Brother

The evening of May 27, 2011, was clear and cold for late spring in Moscow, but sunny, as the crowds began arriving at the soccer stadium in the city’s east. A match had been scheduled for that night between Anzhi, a visiting team from Dagestan, and Lokomotiv, a popular Moscow team. The game was set for early evening, so fans could come after work to Lokomotiv’s home stadium.

Anzhi enjoyed crowds of enthusiastic fans, and more than 1,500 of them had arrived at Lokomotiv stadium that evening from Dagestan to see their team play. The visitors — almost entirely men in their 20s and 30s — were from the North Caucasus, where Russia had fought two wars against rebels in Chechnya, so their arrival in Moscow was met with apprehension. When they arrived at the gate, holding tickets, they were not funneled through the normal stadium gates, but through a separate entrance equipped with what looked to be a rather unremarkable metal detector and some city police.

The fans paid little attention to the camera aimed at each of them as they stepped up to the metal detector. The lens was pointed at their faces; the camera rapidly captured each face in a green digital frame, identifying each face’s distinctive characteristics, including such features as distance between the eyes. Then the camera went to work, snapping several photographs.

Near the metal detectors sat an operator with a laptop who worked for a company called Ladakom-Service. One window on his screen showed the live camera acquiring facial images; another part of the screen showed the captured images, and a program matched those images with faces in a government passport database. When a match was successful, a photograph appeared along the bottom of the screen with the person’s identification details, providing the government with a current photo and a record of the person’s presence at the stadium that could be used for almost any purpose in the future. No one was ever told that he was being recorded.

These young men from the North Caucasus thought they had traveled to the capital just to see a soccer match; they had also walked into a modern and potentially powerful surveillance system.

By 2011, facial recognition in the hands of Russia’s security services was being used as a tool for tracking not just criminals, but all kinds of people, at any kind of public event, in any public place. The same company, Ladakom-Service, had installed this technology in that same year in the entrance hall of one of the city’s busiest metro stations. The station, Okhotny Ryad, is located a stone’s throw from the Kremlin and around the corner from Russia’s lower house of parliament, next to one of the most heavily trafficked streets in Moscow. Today, as people step on the subway escalator, their faces enter a frame and are captured by video cameras. The images are rapidly sent to security service databases. Here, too, there is no notification that anyone is being recorded.

Alexander Abashin, chief executive of Ladakom-Service, the company that developed the system, is a veteran of Russian military intelligence. For 12 years he has been developing and installing facial-recognition systems at airports, railway stations, and stadiums across Russia. He has become something of a zealot for using facial-recognition technology everywhere as a law enforcement tool — even in schools and apartment buildings. His enthusiasm comes in part from his own experience; when burglars struck his home, he grabbed facial images from surveillance video and used them with government databases to eventually apprehend the thieves. He told us that his system was so advanced that a scan of 10 million images would take no more than seven seconds. The facial images and video are sent to the metro system’s situation room, the Interior and Emergencies ministries, and the Federal Security Service, or FSB.

Abashin’s facial-recognition system is a glimpse into a large and mostly hidden phenomenon that remains a profound legacy of the Soviet experience: the use of engineering to build systems for security services like the Interior Ministry and the FSB. These systems were invented and developed by engineers who knew what the systems could do, but rarely, if ever, questioned the purposes for which they were used.

The Communist Party had a monopoly on power and did not want competition. It imposed rigid conditions on all kinds of people; for engineers, there was pressure to conform to the goals of the party-state and to fulfill its technical needs, no questions asked. To succeed meant to work on projects without questioning the big picture. For many decades, Soviet engineers were schooled intensively in technical subjects but rarely if ever had exposure to the humanities; the breadth of their education was exceedingly narrow.

Unlike medical doctors who were trained in ethics, engineers were not. They were taught to be technical servants of the state. As a result, generations of engineers were trained and worked their entire lives with little understanding of politics. These engineers were focused on the immense technical needs of the Soviet Union and were comfortable with the concept of strict order because it suited their understanding of the mechanical world better than the often unruly reality of freedom. Events that followed the Soviet collapse only solidified their views. Many engineers who had worked in the sprawling military-industrial complex were left high and dry. They suffered greatly from the deep cuts in military spending and were resentful. Frustrated, they became fertile ground for anti-Western sentiment.

Sergei Koval was just such an engineer. In 1973, he graduated from the physics department of the State University of Leningrad and went to work at a research center on acoustics, his specialty. Every day, he entered a secret military institute in Leningrad that was developing communications technology and then went to work for an even more secret laboratory within the military institute to create speech-recognition technology that would allow the KGB to identify people over the telephone. After the Soviet Union’s collapse, Koval and his colleagues privatized their expertise, and their business thrived in the 1990s. The main customer, however, remained the security services — primarily the FSB. He never questioned his work for the secret police, in Soviet times or afterward.

We asked Koval what he thought about the ethics of his work and the fact that regimes around the world are using his technology to suppress dissidents. His reply was emotional: “All this talk about technology catching dissidents is just bullshit,” he insisted. “It’s typical of the kind of psychological warfare the Americans use against their opponents. I think all these arguments about human rights are completely hypocritical.”

Koval proudly listed countries where his company’s speech-recognition technology was already in use: Kazakhstan, Kyrgyzstan, Uzbekistan, and Belarus — all repressive authoritarian regimes in the former Soviet Union — as well as Saudi Arabia, Algeria, Yemen, and Turkey. He expressed no reservations about whether his technology was being used against journalists, dissidents, and human rights campaigners. “What can we do about it?” he said. “We just come up with the hardware. It’s just technology that is developed with law enforcement in mind. Sure, you can use it against the good guys just as easily as you can use it against the bad guys. One way or another, these governments will be able to use surveillance technology, whether we supply it or not.… If governments listen in on people’s conversations, it’s not the microphone’s fault!”

When Stalin’s security services in the 1930s and 1940s needed to conduct secret research in particular areas, rather than hiring the talent they needed, they simply arrested scientists and engineers and sent them to special installations, known as sharashkas, which were closed off from the outside and heavily guarded. Under Stalin, the scientists and engineers were motivated to produce quick results, under the threat of being sent to labor camps if they failed. But in the years following Stalin’s death in 1953, this system — under which engineers were kept under close government watch in military-style environments — never fully faded away. Rather, it evolved into a far-reaching network of research institutes. The result was that many thousands of Soviet engineers continued working in security or military research.

It was a Swedish scientist, Gunnar Fant, who in 1960 published a monograph, Acoustic Theory of Speech Production, based on his research at the Massachusetts Institute of Technology. He had found a way to slice up a voice recording into samples and then identify them using mathematics and physics. Fant’s discovery, translated into Russian in 1964, led to a surge of secret research into the topic inside the Soviet Union. But while the scientists pursued the development of technology based on his theory, Fant began to be concerned that law enforcement would abuse speech recognition. Fant’s concerns were confirmed in 1970, when FBI Director J. Edgar Hoover went to Stockholm and gave an interview to the local newspaper, Dagens Nyheter, about the prospects for using voice samples to identify terrorists. When the newspaper asked Fant for comments on Hoover’s remarks, Fant cautioned that the method he had developed was imprecise and that it was premature to use it to identify anyone. His rebuttal was so surprising that photographs of Hoover and Fant opposite each other were printed on the front page, presenting him, as Fant put it later, as a “possible FBI enemy number one.”

Soviet scientists had no such reservations. Research centers working on speech recognition opened in many cities, and the section on acoustics at the national Academy of Sciences coordinated the nationwide research. But everybody knew that the true boss was the KGB.

An instrumental part of the research took place in Leningrad at the Scientific Research Institute of Dalny Svyazi, or long-distance communications, known as Dalsvyaz. It was there that Koval began work in 1973 on acoustics. He was always interested in the science of sound — but the promised monthly salary bonus of 15 percent didn’t hurt either.

The institute was shrouded in secrecy. The offices of his applied acoustics unit were always guarded by men with automatic weapons. The institute, with more than 10,000 personnel, was overseen by a ministry for industrial telecommunications, but its real purpose was to work for the military. It was top secret — and harbored even more secrets inside. The applied acoustics unit of 300 people that Koval joined was not under the control of the institute at all but was instead run by the KGB. It was a classic Russian matryoshka — secrets within secrets. By the 1970s, Koval’s applied acoustic unit became the main coordinator of KGB-funded speech-recognition research. An area of research that began in the 1940s as seven people in an acoustics laboratory had, by the 1970s, become a sprawling, well-funded empire.

This empire cracked with the collapse of the Soviet Union, but it didn’t go away. Initially the KGB cut its research programs. “In 1990 our funding stopped,” Koval recalled, who himself left Dalsvyaz. “Two-thirds of employees quit immediately.” But Koval did not leave the field. With his laboratory chief, Mikhail Khitrov, and five colleagues, he founded a private company that in 1993 became the Speech Technology Center, trademarked in the United States as SpeechPro. They tried to succeed with civilian contracts; for example, they developed a talking book for the Society for the Blind.

But soon their old friends, the security services, returned. Koval’s company got its first contract from the Interior Ministry to build a system for using phonoscopy to identify criminals. Then the FSB offered a contract to make a system that would separate voice from background noise. By the early 2000s the company employed up to 350 people, roughly the size of the original Soviet unit in Dalsvyaz. “I cannot say what kind of work we do for them,” Koval said of the security services, “but it all continues — it’s the same. What we did then, we do now.” The company has developed technology it considers unique in its capability and reach, able to store biometric data ranging from voice samples to photo images and match them to individuals. The voice-recognition technology can identify the speaker, regardless of language, accent, or dialect, based on characteristics of the person’s voice.

On a cold and snowy day in January 2012, in an almost empty cafe near the Chernyshevskaya metro station in St. Petersburg, Koval enthusiastically recalled the story of his company. Koval’s confidence had recently been bolstered by an investment from a source even closer to then-Prime Minister Vladimir Putin than the secret services. In September 2011 Gazprombank — part of the vast business empire of Yuri Kovalchuk, a close friend of Putin — acquired 35 percent of SpeechPro.

Many scientists and engineers from across the Soviet space shared a career trajectory similar to Koval’s, and the path created a similar mindset among them as a group when it came to the use of their inventions by the state. Loren Graham, a preeminent historian of Soviet and Russian science at the Massachusetts Institute of Technology, told us, “Russian scientists and engineers are, on the whole, less interested in the ethical and moral problems of their work than many of their counterparts in Western countries.”

“In the Soviet period, Russian scientists and engineers learned early on that if they raised ethical and moral issues, that this was seen by the authorities as ‘political opposition’ and they would be punished for raising such issues. Therefore, they learned to stay silent, and after a while this silence became ingrained and even a part of their professional definition. Of course, the Soviet Union is long gone, but these attitudes have largely continued.”

In addition, he said, “Engineering education in Russia has been focused on technical issues, with very little attention to larger human, ethical, and moral questions. Engineering education in the U.S. faces similar issues, he said. But, he said, “it is worth noticing that at top engineering schools in the United States, such as the Massachusetts Institute of Technology — my university — every student during four years of engineering study is required to take eight courses, usually one each semester, in the humanities and social sciences. These courses open up deep questions of ethics and ‘meaning,’ which are not considered in technical courses.… [This] has important effects, leading to questions about the social responsibilities of scientists and engineers. And many of the best engineering schools in the United States also have departments of science, technology, and society [STS], where these problems are studied.”

Anatoly Levenchuk, an engineer who, in the early 1990s, helped launch Relcom, the first Internet service provider in Russia, told us that he “tell[s] my students not to apply systems engineering when you work for the government.” Why? “It could be very dangerous. You need to know humanities to deal with the state. If you apply only engineering, you will build a prison as a result. Say you are tasked to address threats.… The best way to address them as engineer is to build a box, a prison. You just close everything off.”

Levenchuk himself ceased to cooperate with the government in 2006 and focused on teaching engineering at Moscow Institute of Physics and Technology, the major university for training engineers for the Russian nuclear industry. Levenchuk sought to present new ideas in his classroom, challenging the traditional rigid approach and urging students to be more open-minded and politically aware. He soon came under attack. In April 2013, an open letter was published on LiveJournal.com, accusing Levenchuk of teaching “fascist” philosophy and values. Levenchuk was accused of systematic destruction of “the Soviet school of design.” The letter then went on to demand that members of the State Duma initiate a request to the Prosecutor General’s Office to investigate whether Levenchuk was a foreign agent, as his lectures contained “pro-Western ideas.” The letter was a sign that the Soviet engineering mindset — the rigid adherence to the technical — was resurging.

On the outskirts of St. Petersburg, a glossy new business center houses a small company named Protei. The company’s office was a bit chaotic when we visited in 2011 — they had just moved in — with tables and wires all over the place. The computers had yet to be installed, but Protei was already making something highly desired outside Russia: the equipment for making sure that the technologies known as SORM, which intercept all kinds of traffic — from phone conversations to the Internet — would work in countries like Uzbekistan and Kazakhstan, where authoritarian rulers with miserable human rights practices and a distinct intolerance for democracy and dissent were eager to use them. In December 2011, WikiLeaks and Privacy International launched the Spy Files project, a database of companies that sell such surveillance gear around the world. Although most of the vendors are British, Israeli, German, and American companies, it also included Koval’s SpeechPro and Protei.

Vadim Sekeresh is another engineer who had made it in the world of security services and secret surveillance. In 2011, Sekeresh was head of the SORM department at Protei. A phlegmatic, 40-year-old graduate of the applied mathematics department of St. Petersburg State University, he seemed unruffled by the WikiLeaks disclosure. Like so many other engineers, he did not ask deep moral or ethical questions about how his products were being used. “I didn’t pay any attention to it,” Sekeresh said of the report. “I didn’t really look into it because the whole thing doesn’t bother me. After all, we are not producing the listening devices or bugs. And … we aren’t the only ones producing such tech anyway.” A few months later he sent us an email, “Lots of crimes are solved thanks to technology. It’s obvious that everything could be used to harm, but it’s not related to the producers.”

In other words: We just come up with the hardware.

This article is an edited excerpt from The Red Web: The Struggle Between Russia’s Digital Dictators and the New Online Revolutionaries.

Photo illustration by FP