Canadian legislators proposed 19,600 amendments —almost certainly AI-generated—to a bill in an attempt to delay its adoption. I wrote about many different legislative delaying tactics in A Hacker’s Mind, but this is a new one. [...]
They’re AI warehouse robots. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here. [...]
Hackers are using Facebook advertisements and hijacked pages to promote fake Artificial Intelligence services, such as MidJourney, OpenAI's SORA and ChatGPT-5, and DALL-E, to infect unsuspecting users with password-stealing malware. [...]
Generative artificial intelligence (AI) has captured the imagination of organizations and individuals around the world, and many have already adopted it to help improve workforce productivity, transform customer experiences, and more. When you use a generative AI-based service, you should understand how the information that you enter into the …
Generative artificial intelligence (AI) has captured the imagination of organizations and individuals around the world, and many have already adopted it to help improve workforce productivity, transform customer experiences, and more. When you use a generative AI-based service, you should understand how the information that you enter into the …
The debate over professionalizing software engineers is decades old. (The basic idea is that, like lawyers and architects, there should be some professional licensing requirement for software engineers.) Here’s a law journal article recommending the same idea for AI engineers. This Article proposes another way: professionalizing AI engineering …
This mini-essay was my contribution to a round table on Power and Governance in the Age of AI. It’s nothing I haven’t said here before, but for anyone who hasn’t read my longer essays on the topic, it’s a shorter introduction. The increasingly centralized control …
This is part 3 of a series of posts on securing generative AI. We recommend starting with the overview post Securing generative AI: An introduction to the Generative AI Security Scoping Matrix, which introduces the scoping matrix detailed in this post. This post discusses the considerations when implementing security …
This is part 3 of a series of posts on securing generative AI. We recommend starting with the overview post Securing generative AI: An introduction to the Generative AI Security Scoping Matrix, which introduces the scoping matrix detailed in this post. This post discusses the considerations when implementing security …
Oh, how the mighty have fallen. A decade ago, social media was celebrated for sparking democratic uprisings in the Arab world and beyond. Now front pages are splashed with stories of social platforms’ role in misinformation, business conspiracy, malfeasance, and risks to mental health. In a 2022 survey, Americans …
Enlarge / Some ASCII art of our favorite visual cliche for a hacker. (credit: Getty Images) Researchers have discovered a new way to hack AI assistants that uses a surprisingly old-school method: ASCII art. It turns out that chat-based large language models such as GPT-4 get so distracted trying to …
Enlarge (credit: Aurich Lawson | Getty Images) AI assistants have been widely available for a little more than a year, and they already have access to our most private thoughts and business secrets. People ask them about becoming pregnant or terminating or preventing pregnancy, consult them when considering a divorce …
Researchers have demonstrated that putting words in ASCII art can cause LLMs—GPT-3.5, GPT-4, Gemini, Claude, and Llama2—to ignore their safety instructions. Research paper. [...]
Researchers ran a global prompt hacking competition, and have documented the results in a paper that both gives a lot of good examples and tries to organize a taxonomy of effective prompt injection strategies. It seems as if the most common successful strategy is the “compound instruction attack,” as …
With the world’s focus turning to misinformation, manipulation, and outright propaganda ahead of the 2024 U.S. presidential election, we know that democracy has an AI problem. But we’re learning that AI has a democracy problem, too. Both challenges must be addressed for the sake of democratic …
Researchers have demonstrated a worm that spreads through prompt injection. Details : In one instance, the researchers, acting as attackers, wrote an email including the adversarial text prompt, which “poisons” the database of an email assistant using retrieval-augmented generation (RAG), a way for LLMs to pull in extra data from …
Brave Software is the next company to jump into AI, announcing a new privacy-preserving AI assistant called "Leo" is rolling out on the Android version of its browser through the latest release, version 1.63. [...]
Artificial intelligence (AI) has been billed as the next frontier of humanity: the newly available expanse whose exploration will drive the next era of growth, wealth, and human flourishing. It’s a scary metaphor. Throughout American history, the drive for expansion and the very concept of terrain up for …
New research : LLM Agents can Autonomously Hack Websites Abstract: In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs can now function autonomously as agents. With the …
Simon Willison has been playing with the video processing capabilities of the new Gemini Pro 1.5 model from Google, and it’s really impressive. Which means a lot of scary new video prompt injection attacks. And remember, given the current state of technology, prompt injection attacks are impossible …
Microsoft announced that it caught Chinese, Russian, and Iranian hackers using its AI tools—presumably coding tools—to improve their hacking abilities. From their report : In collaboration with OpenAI, we are sharing threat intelligence showing detected state affiliated adversaries—tracked as Forest Blizzard, Emerald Sleet, Crimson Sandstorm, Charcoal Typhoon …
OpenAI has removed accounts used by state-sponsored threat groups from Iran, North Korea, China, and Russia, that were abusing its artificial intelligence chatbot, ChatGPT. [...]
Enlarge (credit: Getty Images) Threats from malicious cyberactivity are likely to increase as nation-states, financially motivated criminals, and novices increasingly incorporate artificial intelligence into their routines, the UK’s top intelligence agency said. The assessment, from the UK’s Government Communications Headquarters, predicted ransomware will be the biggest threat …
New research into poisoning AI models : The researchers first trained the AI models using supervised learning and then used additional “safety training” methods, including more supervised learning, reinforcement learning, and adversarial training. After this, they checked if the AI still had hidden behaviors. They found that with specific prompts …
You can find them by searching for OpenAI chatbot warning messages, like: “I’m sorry, I cannot provide a response as it goes against OpenAI’s use case policy.” I hadn’t thought about this before: identifying bots by searching for distinctive bot phrases. [...]