Mistral AI Unveils Mistral Small 3.2: Enhanced Capabilities for AI Integration
Mistral AI has unveiled Mistral Small 3.2, an updated version of its 24B parameter open-source model. This release focuses on refining existing capabilities rather than introducing new architectures, aiming for enhanced instruction following, reduced output repetition, and more robust function calling, making it a more reliable tool for developers.
Mistral Small 3.2: A Refined AI Experience
Mistral Small 3.2 (Mistral-Small-3.2-24B-Instruct-2506) builds upon its predecessor, Mistral Small 3.1, with a focus on surgical improvements. While the core architecture remains unchanged, the update introduces targeted refinements to address specific behaviors and enhance overall reliability. This includes better adherence to precise instructions and a significant reduction in repetitive or infinite generations, a common issue with earlier versions when handling complex prompts.
Key Enhancements and Performance Metrics
The improvements in Mistral Small 3.2 are evident across various benchmarks:
- Instruction Following: Internal accuracy rose from 82.75% (Small 3.1) to 84.78% (Small 3.2). External datasets also showed significant gains:
- Wildbench v2: Improved from 55.6% to 65.33%
- Arena Hard v2: More than doubled from 19.56% to 43.1%
- Reduced Repetition: The rate of infinite generations dropped from 2.11% to 1.29%, nearly a twofold reduction.
- Function Calling: The function calling template has been upgraded for more reliable tool-use scenarios, particularly within frameworks like vLLM.
- STEM-Related Performance: Notable improvements were observed in coding and technical benchmarks:
- HumanEval Plus Pass@5: Increased from 88.99% to 92.90%
- MBPP Pass@5: Rose from 74.63% to 78.33%
- MMLU Pro: Improved from 66.76% to 69.06%
Accessibility and Strategic Positioning
Mistral Small 3.2, like its predecessor, is available under the Apache 2.0 license via Hugging Face. It requires approximately 55 GB of GPU RAM for bf16 or fp16 precision. While Small 3.1 is integrated into platforms like Google Cloud Vertex AI and scheduled for deployment on NVIDIA NIM and Microsoft Azure, Small 3.2 currently offers self-serve access and direct deployment.
This update underscores Mistral AI's commitment to iterative refinement. Although not a complete overhaul, the enhanced reliability and task handling, especially in instruction precision and tool usage, offer a cleaner user experience for developers. The model's French origin and compliance with EU regulations like GDPR and the EU AI Act also make it an attractive option for enterprises operating in Europe.
Sources
