OpenAI o1 vs Anthropic Claude 3.5: Reasoning Capabilities

Coding precision and reasoning benchmarks

Anthropic’s Claude Code gained popularity after Claude 3.5 Sonnet’s coding performance drove usage at the startup Cursor. Claude 3.5 Sonnet reached 49% on SWE-bench Verified for complex debugging and feature implementation, while Claude Opus 4.6 holds an 80.8% score on that same benchmark. OpenAI o1-preview falls short of these coding results. Anthropic’s Claude Code operates from a computer’s command line with access to all files and applications. OpenAI’s Codex models aim for similar agentic goals, but Anthropic trained its models on messy code repositories from an early stage. I find the gap in coding precision between these models remains the deciding factor for developers. While o1-pro maintains a 19.1 LLM Stats Score, Claude 3.5 Haiku sits at 4.6, and Claude Opus 4.6 successfully completes software tasks that human experts take up to 14.5 hours to finish according to METR. Claude Sonnet 4.6 also shows high capability with a 79.6% score on SWE-bench Verified and 72.5% on the OSWorld benchmark. This 72.5% score is a large increase from the 14.9% score recorded when Anthropic launched computer use sixteen months ago. Claude 3.5 Sonnet achieved 65% on graduate-level questions on the GPQA benchmark and 69.2% on complex multi-tool tasks on the TAU-bench retail benchmark. Anthropic also achieved 70.4% accuracy on technical diagrams and charts on the MMMU benchmark.

Pricing and context capacity

The input price for Claude Opus 4.6 is $5.00 per million tokens. This price is 97% cheaper than the $150.00 rate for o1-pro. Output costs for Opus 4.6 are $25.00 per million tokens, whereas o1-pro requires $600.00 per million tokens. Claude Opus 4.6 has a 1,000,000 token context window. This capacity is much larger than the 200,000 token window of o1-pro. I find the large price difference between these models makes o1-pro a difficult choice for high-volume production. Claude Opus 4.6 has more capacity for RAG pipelines and agentic workflows. You should evaluate your specific input-to-output ratio before selecting a provider. OpenAI’s GPT-5.1 Thinking is 7.6x cheaper than o1-preview on a blended 3:1 input/output basis. GPT-5.1 Thinking also has a 400,000 token context window. This is double the 200,000 token capacity of o1-pro. OpenAI’s o1-preview costs $15.00 per million input tokens and $60.00 per million output tokens. Gemini 3.1 Pro costs approximately $2 per million input tokens and $12 per million output tokens for up to 200K context. DeepSeek R1 costs a third of what it takes to use OpenAI o1.

Reasoning capabilities and task performance

o1-preview and o1-mini achieved 24% accuracy on headline selection tasks. This is a failure compared to the 34% accuracy of GPT-4o and GPT-4o-mini. This shows that reasoning capabilities in o1 do not always translate to tasks requiring editorial judgment. Claude Opus 4.6 completes software tasks that take human experts 14.5 hours to finish, according to METR. Claude Sonnet 4.6 reached 72.5% on the OSWorld benchmark. This is an increase from the 14.9% score recorded when Anthropic launched computer use sixteen months ago. OpenAI’s GPT-5.4 achieved 83% on the GDPval benchmark for professional knowledge work. Claude 3.5 Sonnet achieved 65% on graduate-level questions on the GPQA benchmark and 69.2% on complex multi-tool tasks on the TAU-bench retail benchmark. Anthropic also achieved 70.4% accuracy on technical diagrams and charts on the MMMU benchmark. Gemini 3.1 Pro achieved 77.1% on the ARC-AGI-2 benchmark and 94.3% on the GPQA Diamond test. Will reasoning models eventually bridge the gap in human-like editorial intuition?