Live from Black Hat: Hacking Public Opinion with Renée DiResta
Psychological operations, orﾂ?PsyOps, is a topic I???ve been interested in for a while. It???s aﾂ?blend of social engineering and marketing, both passions of mine. That’s why I found the keynote byﾂ?Renﾃｩeﾂ?DiResta,ﾂ?Research Managerﾂ?at theﾂ?Stanford Internet Observatory, particularly interesting.ﾂ?
The Internet Makes Spreading Information Cheap & Easyﾂ?
Disinformation and propaganda areﾂ?oldﾂ?phenomenaﾂ?that can be traced back to the invention of the printing press ??? and arguably before then.ﾂ?With the advent of theﾂ?Internet, the cost of publishing dropped to zero. There are no hosting costs on certain platforms, butﾂ?especially in the beginning, theﾂ?blogosphere was veryﾂ?decentralized,ﾂ?and it was hard to get people to read your content.ﾂ?With theﾂ?rise of social media,ﾂ?you can share your content and it can become viral. At the same time, content creation becomes easier.ﾂ?All of thisﾂ?eliminates cost barriers andﾂ?gatekeepers.ﾂ?ﾂ?
State Actors ???Hack??? Our Opinionsﾂ?
As social media platforms matured, the algorithms that curate content become more and more sophisticated. They are trying to group people and deliver personalized targeting of content, which allows adversaries to analyze and game the algorithms.ﾂ?ﾂ?
State actors don???t just influence, they start hacking public opinion, which involves fake content producers and fake accounts. They can do this more effectively because they understand the ecosystem extremely well, typically applying one of four tactics, sometimes in combination:ﾂ?ﾂ?ﾂ?
- Distract:ﾂ?Taking attention away from news stories that are detrimental to the state actor
- Persuade:ﾂ?Providing convincing content to sway a target???s opinion
- Entrench:ﾂ?Getting individuals to identify with their peer groups and dig their heels in
- Divide:ﾂ?Pitting groups against each other to spread dissentﾂ?
Architecture of a Modern Information Operationﾂ?ﾂ?
Information operations often create fake public personas, such as journalists, to create content. They then seed it to social media and amplify it through bot accounts to get organic shares among the population. Theﾂ?ultimateﾂ?goalﾂ?is to have mass media pick up the stories and amplify even further.ﾂ?ﾂ?
Many of these campaigns use algorithmic manipulation. The Russian disinformation campaign around the 2016 election only spent $100,000 in advertising, but their real lift came from creating compelling content that people shared organically.ﾂ?ﾂ?
From a defensive perspective, you can look at these operations as a kill chain. You should ask yourself: Which part of the chain can I disrupt to slow or stop the campaign? The last hop to mass media is particularly important.ﾂ?ﾂ?
Telling a Positive Story About Chinaﾂ?ﾂ?
China isﾂ?aﾂ?powerful player in informationﾂ?operations,ﾂ?but we???ll see in a moment that their operations have less impact than Russia???s.ﾂ?However, their network infiltrationﾂ?operations, which can be related to information operations, areﾂ?alreadyﾂ?very advanced.ﾂ?ﾂ?
In a nutshell, the goal of China???s information operations is to ???Tell China???s Story Well???. They are primarily concerned with persuasion, sometimes distraction.ﾂ?For example, during the COVID-19 crisis, China first controlledﾂ?domestic perception, then put out English language posts about WHO praising the Chinese response. They pushed this out on Facebook to ensure they reached large global audiences.ﾂ?They flip back and forth between funny things that people retweet and more aggressive messages.ﾂ?
A Lookﾂ?Intoﾂ?Chinese Information Operationsﾂ?ﾂ?
China has decades of experience inﾂ?bothﾂ?covert and overtﾂ?domesticﾂ?information management.ﾂ?They’re now taking these inward-facing capabilities and employing them outside of their borders.ﾂ?ﾂ?
We can classify their content sources into three categories:ﾂ?ﾂ?
- Light:ﾂ?Official state news outlets
- Grey:ﾂ?Content farms thatﾂ?are not easily attributable to the state andﾂ?push outﾂ?fake political storiesﾂ?
- Dark:ﾂ?Purely online properties that spread disinformationﾂ?
Even though Facebook is banned in China, its content platforms haveﾂ?more than 220 million followers. Theyﾂ?have alsoﾂ?expanded to troll accounts and covert strategies, which have been taken down from Facebook and Twitter in some occasions.ﾂ?ﾂ?
Asﾂ?Western media began to talk about Hong Kong protests,ﾂ?Chineseﾂ?troll accountsﾂ?surfaced, pretending to be Hong Kong citizens,ﾂ?andﾂ?toldﾂ?theﾂ?journalists that they gotﾂ?theﾂ?storyﾂ?completelyﾂ?wrong.ﾂ?However, China lost its Hong Kong bots early in the protests because they were shut down.ﾂ?Research showed that most accounts were not createdﾂ?pre-emptivelyﾂ?butﾂ?as a reaction to aﾂ?crisis.ﾂ?ﾂ?
China is Struggling to Have Real Impact, But They???ll Get Betterﾂ?ﾂ?
The surprising thing was that 92 percentﾂ?of accounts had less than 10 followers.ﾂ?Most tweets didn???t even have aﾂ????like,??? and maximum tweetﾂ?engagement was 3,700.ﾂ?In other words, Chinaﾂ?did a very poor jobﾂ?of getting real people to pick up their content.ﾂ?ﾂ?
While China is good at creating content, they are sloppy at their social media game.ﾂ?China is well resourced, and they???reﾂ?committedﾂ?to improving.ﾂ?At the same time, we shouldn???t overemphasize the impact of the efforts.ﾂ?ﾂ?
Russia???s Game: Entrench and Divideﾂ?ﾂ?
By comparison,ﾂ?Russia is best in classﾂ?when it comes to information operations.ﾂ?They excel at creating agents of influence and manipulating media. They are using network infiltration as one of their tactics, both to hack public influencers and by leaking data to the media.ﾂ?ﾂ?
Russia has the same set of overt and covert media, ranging from light to dark, but it spends a fraction of China???s budget. One example of a covert content source isﾂ?BlackMattersUS, which is officially operated by an American activist but isﾂ?actually runﾂ?by a Russian contractor in St. Petersburg.ﾂ?
Its media outlets have fewer Facebook followers, only in the range of 39 million, but they have a lot more engagement. Russia is much better at segmenting their audience and creating custom content that plays into their narratives, entrenching and dividing their audiences. They are also better at picking the right types of media for the audience and social network, e.g. videos for young millennials.ﾂ?ﾂ?
Russian Memes vs. Chinese Narrativesﾂ?ﾂ?
While China is focusing mainly on creating a certain narrative, Russia focuses much more on memes that convey feelings or a point of view.ﾂ?Much of this content is generated by theﾂ?Internet Research Agency,ﾂ?a Russian content farm that is not officially associated with the governmentﾂ?to create plausible deniability. They focus on social content first, whichﾂ?lends itself to certain types of media.ﾂ?ﾂ?
Memes look at how people feel. They areﾂ?identity-focusedﾂ?andﾂ?entrench people inﾂ?their groups. Contentﾂ?isﾂ?createdﾂ?to reinforceﾂ?their beliefs.ﾂ?By sharing the content, individuals areﾂ?signalingﾂ?membership in their group. Interestingly, theﾂ?IRA does this both on theﾂ?politicalﾂ?left and on the right, splitting the country in two.ﾂ?ﾂ?
Creatingﾂ?Agents of Influenceﾂ?ﾂ?
Russia doesn???t stop with online engagement and shaping opinions. It wants to create agents of influence that go out on the street and conduct activism.ﾂ?When you follow the Internet Research Agency page or like a piece of content, you give the IRA a signal that you???re sympathetic to aﾂ?particular pointﾂ?of view.ﾂ?
What DiResta hasﾂ?observedﾂ?is attempts to recruitﾂ?these peopleﾂ?through a constant outreach, more than you???d typically see from a media outlet.ﾂ?Theyﾂ?offer financial resources and logistical support to turn people into agents of influence, mobilizing them, getting them out into the streets as activists.ﾂ?This happens behind the scenes, in direct messages, not visible if you???re simply looking at the memesﾂ?on social media.ﾂ?ﾂ?
Throwing Hacked Dataﾂ?intoﾂ?the Mixﾂ?
Russia goes one step further, engaging GRU hacking operations in its information campaigns.ﾂ?APT28,ﾂ?also known asﾂ?Fancy Bear, began creating fake Facebook pages years ago when the GRU was experimenting withﾂ?these tactics.ﾂ?ﾂ?
The green circles represent fake public personas, often journalists, that put out geopolitical content on their own fake media sites. They share the content with Western and regional blogs to gain wider distribution. However, the GRU did not have a lot of success with this tactic.ﾂ?ﾂ?
They since modified their tactics.ﾂ?Public officials or agencies are hacked,ﾂ?then the material is offered to journalists through fake personas, such asﾂ?Gucciferﾂ?2.0.ﾂ?Theﾂ?Internet Research Agencyﾂ?then createsﾂ?memesﾂ?based on the content to amplify on social media.ﾂ?Finally,ﾂ?RT and Sputnik,ﾂ?Russia???sﾂ?state news outlets,ﾂ?talk about the substance of hack while denyingﾂ?their state???s involvement.ﾂ?ﾂ?
While China is focused on telling a positive story about their country, Russia is more interested in exploiting divisions in our society and using vulnerabilities in our information ecosystem.ﾂ?ﾂ?
Russia Will Use the Same Methods in the 2020 Electionsﾂ?
We should expect Russia to employﾂ?similarﾂ?tactics in the 2020 U.S. presidential elections:ﾂ?ﾂ?
- Hackingﾂ?&ﾂ?leakingﾂ?operationsﾂ?
- Hackingvotingﾂ?machinesﾂ?
- Infiltratinggroupsﾂ?
- Amplifyingnarrativesﾂ?ﾂ?
Even if Russiaﾂ?doesn’tﾂ?hack the voting machines, just claimingﾂ?they’veﾂ?beenﾂ?successfulﾂ?will cause mistrust in the elections. And that is theirﾂ?goal: undermining confidence in our political system.ﾂ?ﾂ?
The Effects Will Outlast Active Operationsﾂ?ﾂ?
You can???t hack a social system if the system is resistant to the attack, but our country is divided and very vulnerable. DiResta found an activist???s page on Facebook that contained 40 percent IRA content. However, the person behind the page was real, not a bot. They were sharing the content because the IRA had created messages that resonated extremely well.ﾂ?ﾂ?ﾂ?
People internalizeﾂ?opinionsﾂ?based on repetition. False stories areﾂ?memorizedﾂ?by real people and spread long after activeﾂ?operationsﾂ?have ceased. We???re all more instrumented than ever before.ﾂ?ﾂ?
The challenge for scientific research is:ﾂ?We canﾂ?easilyﾂ?quantify likes and retweets and see how they are reacting, butﾂ?it???s hard toﾂ?see if it changed hearts and minds.ﾂ?ﾂ?
What Does This Mean for Corporate Information Security?ﾂ?ﾂ?
If you???re a CISO in a company with international competitors,ﾂ?you’reﾂ?just as much at risk. Companiesﾂ?with geopolitical aspects such asﾂ?fracking for oilﾂ?andﾂ?agricultural firms likeﾂ?Monsanto haveﾂ?alreadyﾂ?been targets.ﾂ?Companiesﾂ?taking part in social issuesﾂ?have seenﾂ?contentﾂ?against them amplified on social media.ﾂ?ﾂ?
However, most companies don???t have a position on the org chart to deal with adversarial information operations. As aﾂ?CISO, you probably need to start thinking about how you would respond.ﾂ?But the question isn???t purelyﾂ?technical. It’s not a social media analysis problem.ﾂ?Youﾂ?need toﾂ?conduct red teamingﾂ?exercisesﾂ?that involve people from both technical teams and corporate communications.ﾂ?ﾂ?
ﾂ?
If you found this post interesting and would like an overview of additional Black Hat sessions, visit the Veracode blog.
ﾂ?
*** This is a Security Bloggers Network syndicated blog from Application Security Research, News, and Education Blog authored by [email protected] (ckirsch). Read the original post at: https://www.veracode.com/blog/research/live-black-hat-hacking-public-opinion-renee-diresta