GPT-5 Is Here!

OpenAI's GPT-5 and o3 dominate our AI for DTC benchmarks this month with near perfect 0.91 scores, GPT-5 Mini shines as a cost effective and highly performant option, and surprising mid-tier performers like DeepSeek offer impressive cost-performance ratios.

OpenAI Dominates The New Leaderboard

With GPT-5's August 2025 launch, we re-ran our comprehensive DTC AI benchmarks across 23 of the latest models. The results? GPT-5 claims the crown, but several unexpected models punch way above their weight for common ecommerce tasks.

Our latest benchmark results reveal a fascinating three-tier structure:

Tier 1: The Premium Powerhouses (0.89-0.91)

• GPT-5: 0.91 (tied for #1)

• o3: 0.91 (tied for #1)

• GPT-5 Mini: 0.89 (wow!)

• Claude 3.7 Sonnet: 0.89

Tier 2: Previous Generation AI Still Performs (0.84-0.88)

• Claude Sonnet 4: 0.88

• Claude 3.7 Sonnet:thinking: 0.88

• Gemini 2.5 Pro Preview: 0.87

• GPT-4.1: 0.87

• Multiple DeepSeek variants: 0.84 (consistent across three models)

Tier 3: The Budget Options (0.68-0.82)

• GPT-5 Nano: 0.80

• Mistral Medium 3: 0.80

• Gemini 2.0 Flash Lite: 0.68

GPT-5 Performance: Hype vs Reality

OpenAI claims GPT-5 shows "45% reduction in factual errors compared to GPT-4o" and represents "the best model in the world." Our DTC-specific testing confirms impressive capabilities, but with important nuances:

Where GPT-5 Excels

• Product copywriting: Generates compelling, brand-consistent descriptions

• Email sequence logic: Superior flow mapping and personalization triggers

• Complex segmentation: Handles multi-dimensional customer data analysis

• Error reduction: Significantly fewer hallucinations in product specifications

Where GPT-5 Disappoints

• Cost efficiency: At $1.25 per million input tokens, it's 5x more expensive than mid-tier alternatives

• Speed for volume tasks: Overkill for simple product tagging or basic email variations - it takes a long time to think.

• Nano variant weakness: GPT-5 Nano's 0.80 score suggests aggressive optimization hurt performance. However, GPT-5 Mini strikes a fantastic balance of performance, speed, and price.

Practical Model Selection Guide

Use Premium Models (GPT-5, o3, Claude 4) For:

• Brand voice development and style guides

• Complex email automation workflows

• High-value customer segment analysis

• Creative campaign concepting

Use Mid-Tier Models (GPT-5 Mini, DeepSeek, Gemini 2.5 Pro) For:

• Product description generation at scale

• Basic customer service responses

• Social media content variations

• Standard email template creation

Avoid Entirely:

• Gemini 2.0 Flash Lite (0.68 score proves it's not ready)

• GPT-5 Nano for anything beyond simple classification tasks

Key Takeaways

GPT 5 represents a significant improvement in performance across all our tasks in the eval, edging out competitor models like Claude 3.7/4 Sonnet, Opus, and the top open-source models. GPT-5 Mini represents an impressive combination of performance, speed, and cost. 

We recommend matching model capabilities to task complexity: premium for strategy, mid-tier for execution, and budget for classification to maximize the power of these new technologies while maintaining an appropriate balance between the value of each task and the cost to execute it with each model.

Raleon handles this automatically for you across your key retention tasks – from campaign planning and copywriting, to predictive segmentation and email generation. Try the platform now to experience what AI can do for your brand today.

Benchmark History:

• May 2025 (Claude 4 Update)

• June 2025 (Grok 4 Update)