AI Driving Foreign Influence, Disinformation and Espionage

In early October 2021, director of the NSA and U.S. Cyber Command General Paul Nakasone spoke at the 2021 Mandiant Cyber Defense Summit. In his speech, Nakasone detailed numerous ongoing influence operations and outlined how the entities he commands are tackling nation-state threats. He noted that the main challenge his organizations face can be summed up with, “How can we stay ahead of that?”

There is no doubt advanced language generation capability and artificial intelligence (AI) are of significant value when it comes to production and opex reduction. In the national security arena, however, it’s a different story. Nakasone carries the responsibility for protecting the nation from these threats; in the business arena, CSOs and CISOs should do more to address how AI can be used to create and disseminate disinformation and exploit harvested data to feed AI-enhanced targeting systems.

Generating Disinformation

In May 2021, researchers within the Center for Security and Emerging Technology (CSET) at Georgetown University published an analysis titled Truth, Lies and Automation: How Language Models Could Change Disinformation. The report detailed how cutting-edge AI systems, such as the Generative Pre-Trained Transformer 3 (GPT-3) with its 173 billion machine learning parameters and high-performance natural language generation capabilities, could be used to generate content for those involved in producing disinformation. The authors observe in their executive summary, “In light of this breakthrough, we consider a simple but important question: Can automation generate content for disinformation campaigns? If GPT-3 can write seemingly credible news stories, perhaps it can write compelling fake news stories; if it can draft op-eds, perhaps it can draft misleading tweets.” The analysis notes that creating content for disinformation has long relied on individuals creating content; the use of AI will substantially increase the throughput of those disinformation efforts.

One CSET analyst, senior fellow Andrew Lohn, said at a September 16 CSET event, Can AI Write Disinformation? (video), “We don’t often think of autocomplete as being very capable, but with these large language models, the autocomplete is really capable, and you can tailor what you’re starting with to get it to write all sorts of things.”

AI-Enabled Espionage

To buttress Nakasone’s position, on October 5, 2021, the GAO issued a report on the need for federal agencies to “address foreign influence” within government-funded research and development, especially where participants have potential foreign conflicts of interest. The 17-page report highlights the lack of such guidance and how it increases the possibility that a foreign state-owned enterprise could be double-dipping on the same research. Furthermore, on September 29, 2021, the GAO Research Service provided a report, Federal Cybersecurity: Background Issues for Congress, which explained that, according to the FBI, data from the 2015 Office of Personnel Management breach, in which attackers accessed the federal background investigation files on 21.5 million individuals, was used “for espionage campaigns and to help program artificial intelligence systems.”

The FBI’s analysis has company. In September 2020, Mandiant Threat Intelligence published Pro-PRC Influence Campaign Expands to Dozens of Social Media Platforms, Websites, and Forums in at Least Seven Languages, Attempted to Physically Mobilize Protesters in the U.S. The report explored the depth and breadth of China’s efforts—the campaign was active across 30 social media platforms, supported by 40 websites and distributed in at least seven languages—with similar messaging and calls to action. What role, if any, did Chinese AI capabilities play in orchestrating these identified programmatic efforts?

Nakasone’s concerns are not baseless. The technological capabilities of AI and natural language processing are such that making the distinction between valid information and disinformation is not becoming easier; rather it promises to become even more difficult. CSOs/CISOs should stay alert to the potential for information that could be generated by an unscrupulous competitor to confuse or influence their workforce or customer base—it could be just a keystroke away.

Christopher Burgess

Christopher Burgess (@burgessct) is a writer, speaker and commentator on security issues. He is a former Senior Security Advisor to Cisco and served 30+ years within the CIA which awarded him the Distinguished Career Intelligence Medal upon his retirement. Christopher co-authored the book, “Secrets Stolen, Fortunes Lost, Preventing Intellectual Property Theft and Economic Espionage in the 21st Century”. He also founded the non-profit: Senior Online Safety.

burgesschristopher has 186 posts and counting.See all posts by burgesschristopher