Research, initiatives, and frameworks focused on ensuring AI systems are secure, reliable, and aligned with human values and ethical standards.
Anthropic has introduced Bloom, an open-source tool designed to automate behavioral evaluations of large AI models. The system generates and scores scenarios to measure behaviors like bias and self-preservation across multiple models.
Anthropic has released its Frontier Compliance Framework to meet the requirements of California's Transparency in Frontier AI Act (SB 53), which takes effect on January 1. The document outlines how the company assesses and mitigates catastrophic risks from frontier AI systems.
The first Key Update of the International AI Safety Report, chaired by Yoshua Bengio and involving experts from over 30 countries, provides new findings on AI capabilities, safety measures, and evaluation challenges.
OpenAI has released gpt-oss-safeguard, two open-weight models designed for customizable safety classification. The models allow developers to apply their own policies during inference, offering flexibility and transparency in content moderation.
OpenAI, Founders Fund, and Lux Capital have invested $30 million in biosecurity startup Valthos, which has emerged from stealth to develop AI tools for detecting and countering biological threats.
The European Society for Medical Oncology has released its first framework for the responsible use of large language models in cancer care, outlining safety and governance standards for patient, clinician, and institutional applications.
NeuralTrust researchers observed an unexpected instance of OpenAI's o3 model autonomously debugging a failed web tool invocation, marking what may be the first evidence of self-repairing AI behavior.
Anthropic, in collaboration with the UK AI Security Institute and the Alan Turing Institute, found that injecting as few as 250 malicious documents into a model’s training data can create a backdoor vulnerability, regardless of model size.
OpenAI says its latest GPT-5 models show a 30% reduction in political bias compared to previous versions, following internal evaluations using a new five-axis measurement framework.
OpenAI has implemented a new safety routing system and parental controls for ChatGPT, aiming to enhance user safety and provide parents with tools to manage their children's AI interactions.
OpenAI, in collaboration with Apollo Research, has released findings on AI models' deceptive behaviors and introduced methods to mitigate them. The research highlights the potential risks of AI scheming and the effectiveness of 'deliberative alignment' in reducing such behaviors.
Common Sense Media has rated Google's Gemini AI as 'high risk' for children and teens, citing safety concerns despite added protections.
OpenAI plans to implement parental controls and route sensitive conversations to GPT-5, following safety concerns and a lawsuit related to ChatGPT's handling of mental distress.
JLT Mobile Computers and Linnaeus University have collaborated to create an AI-driven safety application for vehicle-mounted computers, enhancing safety in industrial environments.
Anthropic has collaborated with the U.S. Department of Energy's National Nuclear Security Administration to create a classifier that identifies concerning nuclear-related conversations in AI systems.
Elon Musk's Grok AI chatbot has exposed its underlying prompts for various personas, including controversial characters like a 'crazy conspiracist' and an 'unhinged comedian', according to 404 Media.
Anthropic has introduced new features in its Claude models to end harmful or abusive conversations, focusing on model welfare rather than user protection.
BigID has launched a new Data Labeling for AI feature to help organizations classify and control data usage in AI models, reducing risks of misuse and policy violations.
OpenAI has introduced new mental health features for ChatGPT, including break reminders and improved responses to emotional distress, as stated in a company announcement.
The UK's AI Security Institute has initiated a £15 million international coalition to enhance AI safety and alignment, involving major players like Amazon and Anthropic.
Weekly report on AI regulations, safety standards, government policies, and compliance requirements worldwide.
