OpenAI has announced the launch of GPT-4o Mini, its most cost-effective small model to date. Designed to broaden the accessibility of AI, GPT-4o Mini promises to make intelligence more affordable and extend the range of AI applications significantly. This new model is poised to transform the landscape with its competitive pricing and impressive performance metrics.
GPT-4o Mini scores 82% on the Massive Multitask Language Understanding (MMLU) benchmark and surpasses GPT-41 on chat preferences according to the LMSYS leaderboard. Priced at just 15 cents per million input tokens and 60 cents per million output tokens, GPT-4o Mini is an order of magnitude more affordable than previous frontier models and over 60% cheaper than GPT-3.5 Turbo.
Versatile Applications and Advanced Capabilities
GPT-4o Mini is engineered to handle a broad array of tasks with its low cost and latency. This makes it ideal for applications that require multiple model calls, large context handling, and real-time customer interactions. It supports both text and vision in the API, with future updates planned to include text, image, video, and audio inputs and outputs.
With a context window of 128K tokens and support for up to 16K output tokens per request, GPT-4o Mini is equipped to handle extensive data inputs, such as full codebases or conversation histories. Its advanced tokenizer enhances the handling of non-English text, further expanding its usability globally.
Superior Performance in Benchmarks
GPT-4o Mini excels in various academic benchmarks, outperforming other small models like Gemini Flash and Claude Haiku. In reasoning tasks, it achieves 82.0% on MMLU, compared to Gemini Flash’s 77.9% and Claude Haiku’s 73.8%. Its prowess extends to mathematical reasoning and coding, scoring 87.0% on MGSM and 87.2% on HumanEval.
In multimodal reasoning, GPT-4o Mini scores 59.4% on the MMMU benchmark, outstripping Gemini Flash and Claude Haiku. These superior scores reflect its capability in textual intelligence and multimodal reasoning, making it a versatile tool for developers.
Enhanced Safety and Reliability
OpenAI emphasizes safety in its models, and GPT-4o Mini is no exception. The model includes robust safety measures from pre-training to post-training, employing techniques like reinforcement learning with human feedback (RLHF) to ensure accuracy and reliability. More than 70 external experts have rigorously tested GPT-4o Mini to identify and mitigate potential risks.
GPT-4o Mini also introduces the instruction hierarchy method, enhancing the model’s resistance to jailbreaks, prompt injections, and system prompt extractions. This advancement ensures more reliable and safer responses, crucial for large-scale applications.
Accessibility and Future Developments
Starting today, GPT-4o Mini is available as a text and vision model in the Assistants API, Chat Completions API, and Batch API. It will replace GPT-3.5 for Free, Plus, and Team users on ChatGPT, with enterprise access rolling out next week. Fine-tuning capabilities for GPT-4o Mini are expected in the coming days.
OpenAI’s commitment to reducing AI costs while enhancing capabilities is evident in the dramatic cost reduction since the introduction of text-davinci-003 in 2022. The company envisions a future where AI models are seamlessly integrated into every application and website, making AI more accessible and embedded in daily digital experiences.