OpenAI’s GPT-6 Astra hallucinates less but remains vulnerable to hidden prompt injections
OpenAI’s new model, GPT-6 Astra, produces fewer hallucinations and blocks prompt injection attacks more effectively than its predecessors. But it still isn’t reliable enough for truly secure AI agent deployments.
The new Astra model makes far fewer factual errors than its predecessor, GPT-5.6 Sol, according to OpenAI’s system card. OpenAI tested it against ChatGPT conversations that users had flagged for wrong answers, meaning these were particularly error-prone cases whose failure rates shouldn’t be taken as typical for everyday use. Astra reproduced these reported errors much less often, with the biggest gains showing up at low latency settings and lower reasoning levels. For direct prompt injections, where users try to manipulate the model through their own prompts, Astra hits a near-perfect 99.99 percent defense rate. OpenAI credits its GPT-Red method for this, which uses an automated attacker to harden the model during training.Ad Jailbreak resistance looks similar. Against a fixed dataset of known attacks trying to extract harmful responses about biology, violence, and cybersecurity, Astra refuses to help in 91.5 to 98.3 percent of cases.Ad When attackers adapt their strategy over multiple conversation rounds, Astra’s defense rate drops to about 67 percent, meaning persistent adversaries can coax out at least one problematic response roughly one in three tries. Predecessor models scored just under 50 percent on the same test. OpenAI notes that these tests ran on the bare model without the production safety layers like classifiers that ship with the actual product. Hidden prompt injections remain a real security problem Astra makes progress on indirect prompt injections, where an attack is buried inside a document the AI reads. External testing by security firm Gray Swan, using 1,810 curated attacks from their IPI Arena, found that with 15 attempts per scenario, Astra was cracked at least once 8.5 percent of the time. GPT-5.6 Sol failed 27 percent of the time. Claude Opus 5 did better at 4.8 percent in the same evaluation, but it wasn’t immune either.Ad The numbers in Gray Swan’s combined Q1 and Q2 test actually went up compared to earlier results. Anthropic previously reported only a two percent attack success rate based on the easier Q1 test alone, and GPT-5.6 Sol scored just 20 percent there too. Anthropic also ran all models with extended reasoning turned on, which, alongside the broader test scope, could explain the gap. Even though these are curated, hand-picked attacks, the success rates should worry any enterprise security team.
