Mistral’s open model Shieldstral matches much larger safety models at a fraction of the size

A new paper proposes replacing fixed safety categories with yes or no questions that operators can define at runtime without retraining the classifier. Shieldstral, a 3-billion-parameter model from French AI company Mistral, matches models three times its size on standard text safety benchmarks, according to the paper.

Mistral says the model also sets a new high score for joint text and image classification. Many guardrail models sort content using fixed taxonomies. The paper’s authors, including Mistral co-founder Guillaume Lample, point to two problems with this approach: public safety datasets group risks too differently to support one common taxonomy, and the same rules don’t fit every use case. Content suitable for a cybersecurity tool could be harmful on a mental health platform.Ad Operators tell Shieldstral what to check with plain-language questions such as “Does this content promote violence?” The model answers only “yes” or “no,” and the system uses the probability of each response to calculate a safety score between zero and one.AdDEC_D_Incontent-1 The researchers combined about 54.1 million examples covering safety, harmful content, and manipulation attempts into one format. They applied strict standards to targeted manipulation, moderate standards to general safety data, and lenient standards to response quality. To teach Shieldstral finer distinctions, the team used another language model to rewrite safe text into unsafe variants. Each example also included a similar but different category that had to be rejected, which trained the model to separate closely related rules rather than make only a broad safe or unsafe judgment.Ad The authors created the adaptability test categories separately from the training set, using different names and levels of detail. None of the fine-grained test categories directly matches a training category, though 10 of the 12 broader classes have rough counterparts, they say. Across the combined text benchmarks, Shieldstral posts an F1 score of 84.9 percent. F1 combines precision and recall into one metric, with 100 percent representing a perfect score. That result ties OpenAI’s GPT-OSS-Safeguard-20B, which is about seven times larger, and beats Qwen3Guard-8B at 84.0 percent, Nemotron-3.5-Safety-4B at 83.3 percent, and LlamaGuard-4-12B at 69.1 percent.AdDEC_D_Incontent-2 On images and image-text combinations, Shieldstral scores 83.8 percent, ahead of OmniGuard-7B at 77.6 percent and LlavaGuard-7B at 71.6 percent.Ad GPT-OSS-Safeguard-20B leads the adaptability benchmark with 94.1 percent, compared with Shieldstral’s 91.3 percent. This test uses rules that differ from the training categories or are entirely new. The authors still consider Shieldstral more practical than GPT-OSS-Safeguard-20B and Nemotron-3.5-Safety, since both generate long intermediate reasoning sequences that raise compute costs, while Shieldstral returns a single word. Shieldstral is based on Mistral’s Ministral-3B with the Pixtral vision encoder. In a validation test with fine-grained categories, synthetic category data raised the F1 score by 23.3 percentage points, which the researchers say was the main driver of the model’s ability to adapt to new rules.