academic papers
Licensing AI Engineers
The debate over professionalizing software engineers is decades old. (The basic idea is that, like lawyers and architects, there should be some professional licensing requirement for software engineers.) Here’s a law journal ...
A Taxonomy of Prompt Injection Attacks
Researchers ran a global prompt hacking competition, and have documented the results in a paper that both gives a lot of good examples and tries to organize a taxonomy of effective prompt ...
LLM Prompt Injection Worm
Researchers have demonstrated a worm that spreads through prompt injection. Details: In one instance, the researchers, acting as attackers, wrote an email including the adversarial text prompt, which “poisons” the database of ...
On Software Liabilities
Over on Lawfare, Jim Dempsey published a really interesting proposal for software liability: “Standard for Software Liability: Focus on the Product for Liability, Focus on the Process for Safe Harbor.” Section 1 ...
Teaching LLMs to Be Deceptive
Interesting research: “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training“: Abstract: Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order ...
Poisoning AI Models
New research into poisoning AI models: The researchers first trained the AI models using supervised learning and then used additional “safety training” methods, including more supervised learning, reinforcement learning, and adversarial training ...
Side Channels Are Common
Really interesting research: “Lend Me Your Ear: Passive Remote Physical Side Channels on PCs.” Abstract: We show that built-in sensors in commodity PCs, such as microphones, inadvertently capture electromagnetic side-channel leakage from ...
Improving Shor’s Algorithm
We don’t have a useful quantum computer yet, but we do have quantum algorithms. Shor’s algorithm has the potential to factor large numbers faster than otherwise possible, which—if the run times are ...
Security Analysis of a Thirteenth-Century Venetian Election Protocol
Interesting analysis: This paper discusses the protocol used for electing the Doge of Venice between 1268 and the end of the Republic in 1797. We will show that it has some useful ...
AI Decides to Engage in Insider Trading
A stock-trading AI (a simulated experiment) engaged in insider trading, even though it “knew” it was wrong. The agent is put under pressure in three ways. First, it receives a email from ...
