OpenAI's containerized ChatGPT environment is open to limited yet extensive access to core instructions while allowing arbitrary file uploads and command execution within the isolated sandbox. [...]
Google is adding a new AI-powered scam protection feature that monitors phone call conversations on Google Pixel devices to detect patterns that warn when the caller may be a scammer. [...]
The Open Source Initiative has published (news article here ) its definition of “open source AI,” and it’s terrible. It allows for secret training data and mechanisms. It allows for development to be done in secret. Since for a neural network, the training data is the source code—it …
Interesting research: “ Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks “: Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense strategy tailored to counter LLM-driven cyberattacks. We introduce Mantis, a …
Really interesting research: “ An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection “: Abstract : Large Language Models (LLMs) have transformed code com- pletion tasks, providing context-based suggestions to boost developer productivity in software engineering. As users often fine-tune these models for specific applications, poisoning …
I’ve been writing about the possibility of AIs automatically discovering code vulnerabilities since at least 2018. This is an ongoing area of research: AIs doing source code scanning, AIs finding zero-days in the wild, and everything in between. The AIs aren’t very good at it yet, but …
Researchers have shown that it's possible to abuse OpenAI's real-time voice API for ChatGPT-4o, an advanced LLM chatbot, to conduct financial scams with low to moderate success rates. [...]
OpenAI's new "ChatGPT search" Chrome extension feels like nothing more than a typical search hijacker, changing Chrome's settings so your address bar searches go through ChatGPT Search instead. [...]
Researchers at Google have developed a watermark for LLM-generated text. The basics are pretty obvious: the LLM chooses between tokens partly based on a cryptographic key, and someone with knowledge of the key can detect those choices. What makes this hard is (1) how much text is required for …
Tax farming is the practice of licensing tax collection to private contractors. Used heavily in ancient Rome, it’s largely fallen out of practice because of the obvious conflict of interest between the state and the contractor. Because tax farmers are primarily interested in short-term revenue, they have no …
OpenAI has disrupted over 20 malicious cyber operations abusing its AI-powered chatbot, ChatGPT, for debugging and developing malware, spreading misinformation, evading detection, and conducting spear-phishing attacks. [...]
In July, I wrote about my new book project on AI and democracy, to be published by MIT Press in fall 2025. My co-author and collaborator Nathan Sanders and I are hard at work writing. At this point, we would like feedback on titles. Here are four possibilities: Rewiring …
An Australian news agency is reporting that robot vacuum cleaners from the Chinese company Deebot are surreptitiously taking photos and recording audio, and sending that data back to the vendor to train their AIs. Ecovacs’s privacy policy— available elsewhere in the app —allows for blanket collection of user …
The notorious APT hacking group known as FIN7 launched a network of fake AI-powered deepnude generator sites to infect visitors with information-stealing malware. [...]
Governor Newsom has vetoed the state’s AI safety bill. I have mixed feelings about the bill. There’s a lot to like about it, and I want governments to regulate in this space. But, for now, it’s all EU. (Related, the Council of Europe treaty on AI …
This vulnerability hacks a feature that allows ChatGPT to have long-term memory, where it uses information from past conversations to inform future conversations with that same user. A researcher found that he could use that feature to plant “false memories” into that context window that could subvert the model …
For years now, AI has undermined the public’s ability to trust what it sees, hears, and reads. The Republican National Committee released a provocative ad offering an “AI-generated look into the country’s possible future if Joe Biden is re-elected,” showing apocalyptic, machine-made images of ruined cityscapes and …
A critical vulnerability in NVIDIA Container Toolkit impacts all AI applications in a cloud or on-premise environment that rely on it to access GPU resources. [...]
While cybercriminals have used generative AI technology to create convincing emails, government agencies have warned about the potential abuse of AI tools to creating malicious software, despite the safeguards and restrictions that vendors implemented. [...]
Over the summer, I gave a talk about AI and democracy at TedXBillings. The recording is https://www.youtube.com/watch?v=uqC4nb7fLpY”>live. Please share. I’m hoping for more than 200 views.... [...]
New research evaluating the effectiveness of reward modeling during Reinforcement Learning from Human Feedback (RLHF): “ SEAL: Systematic Error Analysis for Value ALignment.” The paper introduces quantitative metrics for evaluating the effectiveness of modeling and aligning human values: Abstract : Reinforcement Learning from Human Feedback (RLHF) aims to align language models …
Microsoft announced today that it has partnered with StopNCII to proactively remove harmful intimate images and videos from Bing using digital hashes people create from their sensitive media. [...]
North Carolina musician Michael Smith was indicted for collecting over $10 million in royalty payments from Spotify, Amazon Music, Apple Music, and YouTube Music using AI-generated songs streamed by thousands of bots in a massive streaming fraud scheme. [...]
Interesting paper: “ Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data “: Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However …
Yet another SQUID acronym : SQUID, short for Surrogate Quantitative Interpretability for Deepnets, is a computational tool created by Cold Spring Harbor Laboratory (CSHL) scientists. It’s designed to help interpret how AI models analyze the genome. Compared with other analysis tools, SQUID is more consistent, reduces background noise, and …