Live from Black Hat: Hacking Public Opinion with Renée DiResta 

Psychological operations, orツ?PsyOps, is a topic I???ve been interested in for a while. It???s aツ?blend of social engineering and marketing, both passions of mine. That’s why I found the keynote byツ?Renテゥeツ?DiResta,ツ?Research Managerツ?at theツ?Stanford Internet Observatory, particularly interesting.ツ?

The Internet Makes Spreading Information Cheap & Easyツ?

Disinformation and propaganda areツ?oldツ?phenomenaツ?that can be traced back to the invention of the printing press ??? and arguably before then.ツ?With the advent of theツ?Internet, the cost of publishing dropped to zero. There are no hosting costs on certain platforms, butツ?especially in the beginning, theツ?blogosphere was veryツ?decentralized,ツ?and it was hard to get people to read your content.ツ?With theツ?rise of social media,ツ?you can share your content and it can become viral. At the same time, content creation becomes easier.ツ?All of thisツ?eliminates cost barriers andツ?gatekeepers.ツ?ツ?

State Actors ???Hack??? Our Opinionsツ?

As social media platforms matured, the algorithms that curate content become more and more sophisticated. They are trying to group people and deliver personalized targeting of content, which allows adversaries to analyze and game the algorithms.ツ?ツ?


State actors don???t just influence, they start hacking public opinion, which involves fake content producers and fake accounts. They can do this more effectively because they understand the ecosystem extremely well, typically applying one of four tactics, sometimes in combination:ツ?ツ?ツ?

  • Distract:ツ?Taking attention away from news stories that are detrimental to the state actor
  • Persuade:ツ?Providing convincing content to sway a target???s opinion
  • Entrench:ツ?Getting individuals to identify with their peer groups and dig their heels in
  • Divide:ツ?Pitting groups against each other to spread dissentツ?

Architecture of a Modern Information Operationツ?ツ?

Architecture of a Modern Information Operation

Information operations often create fake public personas, such as journalists, to create content. They then seed it to social media and amplify it through bot accounts to get organic shares among the population. Theツ?ultimateツ?goalツ?is to have mass media pick up the stories and amplify even further.ツ?ツ?

Many of these campaigns use algorithmic manipulation. The Russian disinformation campaign around the 2016 election only spent $100,000 in advertising, but their real lift came from creating compelling content that people shared organically.ツ?ツ?

From a defensive perspective, you can look at these operations as a kill chain. You should ask yourself: Which part of the chain can I disrupt to slow or stop the campaign? The last hop to mass media is particularly important.ツ?ツ?

Telling a Positive Story About Chinaツ?ツ?

China isツ?aツ?powerful player in informationツ?operations,ツ?but we???ll see in a moment that their operations have less impact than Russia???s.ツ?However, their network infiltrationツ?operations, which can be related to information operations, areツ?alreadyツ?very advanced.ツ?ツ?

In a nutshell, the goal of China???s information operations is to ???Tell China???s Story Well???. They are primarily concerned with persuasion, sometimes distraction.ツ?For example, during the COVID-19 crisis, China first controlledツ?domestic perception, then put out English language posts about WHO praising the Chinese response. They pushed this out on Facebook to ensure they reached large global audiences.ツ?They flip back and forth between funny things that people retweet and more aggressive messages.ツ?

A Lookツ?Intoツ?Chinese Information Operationsツ?ツ?

China has decades of experience inツ?bothツ?covert and overtツ?domesticツ?information management.ツ?They’re now taking these inward-facing capabilities and employing them outside of their borders.ツ?ツ?

We can classify their content sources into three categories:ツ?ツ?

  • Light:ツ?Official state news outlets
  • Grey:ツ?Content farms thatツ?are not easily attributable to the state andツ?push outツ?fake political storiesツ?
  • Dark:ツ?Purely online properties that spread disinformationツ?

Even though Facebook is banned in China, its content platforms haveツ?more than 220 million followers. Theyツ?have alsoツ?expanded to troll accounts and covert strategies, which have been taken down from Facebook and Twitter in some occasions.ツ?ツ?

Asツ?Western media began to talk about Hong Kong protests,ツ?Chineseツ?troll accountsツ?surfaced, pretending to be Hong Kong citizens,ツ?andツ?toldツ?theツ?journalists that they gotツ?theツ?storyツ?completelyツ?wrong.ツ?However, China lost its Hong Kong bots early in the protests because they were shut down.ツ?Research showed that most accounts were not createdツ?pre-emptivelyツ?butツ?as a reaction to aツ?crisis.ツ?ツ?


China is Struggling to Have Real Impact, But They???ll Get Betterツ?ツ?

The surprising thing was that 92 percentツ?of accounts had less than 10 followers.ツ?Most tweets didn???t even have aツ????like,??? and maximum tweetツ?engagement was 3,700.ツ?In other words, Chinaツ?did a very poor jobツ?of getting real people to pick up their content.ツ?ツ?

While China is good at creating content, they are sloppy at their social media game.ツ?China is well resourced, and they???reツ?committedツ?to improving.ツ?At the same time, we shouldn???t overemphasize the impact of the efforts.ツ?ツ?

Russia???s Game: Entrench and Divideツ?ツ?

By comparison,ツ?Russia is best in classツ?when it comes to information operations.ツ?They excel at creating agents of influence and manipulating media. They are using network infiltration as one of their tactics, both to hack public influencers and by leaking data to the media.ツ?ツ?

Russia has the same set of overt and covert media, ranging from light to dark, but it spends a fraction of China???s budget. One example of a covert content source isツ?BlackMattersUS, which is officially operated by an American activist but isツ?actually runツ?by a Russian contractor in St. Petersburg.ツ?

Its media outlets have fewer Facebook followers, only in the range of 39 million, but they have a lot more engagement. Russia is much better at segmenting their audience and creating custom content that plays into their narratives, entrenching and dividing their audiences. They are also better at picking the right types of media for the audience and social network, e.g. videos for young millennials.ツ?ツ?

Russian Memes vs. Chinese Narrativesツ?ツ?

While China is focusing mainly on creating a certain narrative, Russia focuses much more on memes that convey feelings or a point of view.ツ?Much of this content is generated by theツ?Internet Research Agency,ツ?a Russian content farm that is not officially associated with the governmentツ?to create plausible deniability. They focus on social content first, whichツ?lends itself to certain types of media.ツ?ツ?

Russian meme

Memes look at how people feel. They areツ?identity-focusedツ?andツ?entrench people inツ?their groups. Contentツ?isツ?createdツ?to reinforceツ?their beliefs.ツ?By sharing the content, individuals areツ?signalingツ?membership in their group. Interestingly, theツ?IRA does this both on theツ?politicalツ?left and on the right, splitting the country in two.ツ?ツ?

Creatingツ?Agents of Influenceツ?ツ?

Russia doesn???t stop with online engagement and shaping opinions. It wants to create agents of influence that go out on the street and conduct activism.ツ?When you follow the Internet Research Agency page or like a piece of content, you give the IRA a signal that you???re sympathetic to aツ?particular pointツ?of view.ツ?

What DiResta hasツ?observedツ?is attempts to recruitツ?these peopleツ?through a constant outreach, more than you???d typically see from a media outlet.ツ?Theyツ?offer financial resources and logistical support to turn people into agents of influence, mobilizing them, getting them out into the streets as activists.ツ?This happens behind the scenes, in direct messages, not visible if you???re simply looking at the memesツ?on social media.ツ?ツ?

Throwing Hacked Dataツ?intoツ?the Mixツ?

Russia goes one step further, engaging GRU hacking operations in its information campaigns.ツ?APT28,ツ?also known asツ?Fancy Bear, began creating fake Facebook pages years ago when the GRU was experimenting withツ?these tactics.ツ?ツ?

Hacked data

The green circles represent fake public personas, often journalists, that put out geopolitical content on their own fake media sites. They share the content with Western and regional blogs to gain wider distribution. However, the GRU did not have a lot of success with this tactic.ツ?ツ?

They since modified their tactics.ツ?Public officials or agencies are hacked,ツ?then the material is offered to journalists through fake personas, such asツ?Gucciferツ?2.0.ツ?Theツ?Internet Research Agencyツ?then createsツ?memesツ?based on the content to amplify on social media.ツ?Finally,ツ?RT and Sputnik,ツ?Russia???sツ?state news outlets,ツ?talk about the substance of hack while denyingツ?their state???s involvement.ツ?ツ?

While China is focused on telling a positive story about their country, Russia is more interested in exploiting divisions in our society and using vulnerabilities in our information ecosystem.ツ?ツ?

Russia Will Use the Same Methods in the 2020 Electionsツ?

We should expect Russia to employツ?similarツ?tactics in the 2020 U.S. presidential elections:ツ?ツ?

  • Hackingツ?&ツ?leakingツ?operationsツ?
  • Hackingvotingツ?machinesツ?
  • Infiltratinggroupsツ?
  • Amplifyingnarrativesツ?ツ?

Even if Russiaツ?doesn’tツ?hack the voting machines, just claimingツ?they’veツ?beenツ?successfulツ?will cause mistrust in the elections. And that is theirツ?goal: undermining confidence in our political system.ツ?ツ?

The Effects Will Outlast Active Operationsツ?ツ?

You can???t hack a social system if the system is resistant to the attack, but our country is divided and very vulnerable. DiResta found an activist???s page on Facebook that contained 40 percent IRA content. However, the person behind the page was real, not a bot. They were sharing the content because the IRA had created messages that resonated extremely well.ツ?ツ?ツ?

People internalizeツ?opinionsツ?based on repetition. False stories areツ?memorizedツ?by real people and spread long after activeツ?operationsツ?have ceased. We???re all more instrumented than ever before.ツ?ツ?

The challenge for scientific research is:ツ?We canツ?easilyツ?quantify likes and retweets and see how they are reacting, butツ?it???s hard toツ?see if it changed hearts and minds.ツ?ツ?

What Does This Mean for Corporate Information Security?ツ?ツ?

If you???re a CISO in a company with international competitors,ツ?you’reツ?just as much at risk. Companiesツ?with geopolitical aspects such asツ?fracking for oilツ?andツ?agricultural firms likeツ?Monsanto haveツ?alreadyツ?been targets.ツ?Companiesツ?taking part in social issuesツ?have seenツ?contentツ?against them amplified on social media.ツ?ツ?

However, most companies don???t have a position on the org chart to deal with adversarial information operations. As aツ?CISO, you probably need to start thinking about how you would respond.ツ?But the question isn???t purelyツ?technical. It’s not a social media analysis problem.ツ?Youツ?need toツ?conduct red teamingツ?exercisesツ?that involve people from both technical teams and corporate communications.ツ?ツ?


If you found this post interesting and would like an overview of additional Black Hat sessions, visit the Veracode blog.


*** This is a Security Bloggers Network syndicated blog from Application Security Research, News, and Education Blog authored by [email protected] (ckirsch). Read the original post at: