WithSecure Experiments Highlight Language Model Threat
A report published today by WithSecure (formerly known as F-Secure Business) showed how generative pre-trained transformer language models such as GPT-3 can use machine learning to generate text capable of driving phishing and business email compromise (BEC) campaigns at unprecedented levels of scale.
While language models are typically trained using a specific corpus of data, WithSecure researchers were able to expose these models to additional content that could be used to prompt a specific outcome. For example, an article that made false claims could be used to generate an entire stream of similar harmful or misleading content.
WithSecure researchers used that approach to create a series of experiments involving phishing and spear phishing, harassment, social validation for scams, appropriation of a writing style and the creation of deliberately divisive opinions that could all be used to create malicious text and fake news.
Andy Patel, an intelligence researcher for WithSecure, said as a result, identifying malicious or abusive content is going to become much more difficult. Cybercriminals, for example, will not need to master the nuances of a specific language to launch a phishing campaign. Instead, machine learning algorithms will create grammatically correct content in any language.
In addition, he added that the ability to appropriate a specific writing style will make it much simpler for a phishing attack to mimic the specific writing style of someone an intended victim already trusts.
In theory, at least, text generation platform providers would have policies in place to prevent misuse. However, the WithSecure experiment clearly showed that a seemingly innocuous use of the platform is, in reality, a method for driving highly targeted—and, likely successful—phishing campaigns. No amount of end user training is likely to detect that an email that reads exactly like a legitimate email sent by a senior executive was written by a machine.
Even prior to the rise of platforms such as GPT, cybersecurity teams struggled to keep pace with the existing volume of cyberattacks. The WithSecure report suggested those attacks will increase not just in volume but also in sophistication. Many of the anti-phishing tools that cybersecurity teams rely on today are not going to be able to detect those attacks, noted Patel.
It’s even less clear how and to what degree that capability will impact trust in interpersonal communications—unless an ability to verify the veracity of content is developed and effectively implemented at the point where content is consumed.
It’s impossible to tell how quickly this next wave of cyberattacks might manifest, but organizations are going to need to find additional ways of validating documents. Every communication will need to be more closely examined. Organizations may even need to develop specific countersigns to validate electronic documents. In fact, entirely different approaches to cybersecurity will soon be required.
