In his Import AI newsletter, Clark debunked claims that AI progress is reaching its limits. "Anyone who told you progress is slowing or hitting a wall is wrong," he wrote, highlighting OpenAI’s new o3 model as proof.
Key Insights on o3 and Scaling Innovations:
- OpenAI’s o3 Model: Unlike previous models that rely solely on scaling size, o3 leverages reinforcement learning and additional computational power during execution.
- Dynamic Capabilities: This approach allows the model to "think aloud" during inference, enabling entirely new ways of scaling AI.
- 2025 Trends: Clark predicts that in 2025, companies will combine traditional scaling methods (like larger base models) with innovative techniques to optimize computational power during both training and inference. OpenAI’s o-series models reflect this strategy.
Challenges Ahead:
- Computational Costs: The most advanced version of o3 requires 170 times more computational power than its base version, which already exceeds the demands of o1. This variance in resource needs complicates cost predictions, as tasks can dramatically alter computational requirements.
- Economic Viability: These high costs pose challenges, especially as organizations balance performance improvements with operational expenses.
Clark’s Warning and Anthropic’s Position:
Clark warns that the pace of AI advancements will likely outstrip public expectations. "I don’t think anyone is prepared for just how rapid progress will be from here," he cautioned.
Clark warns that the pace of AI advancements will likely outstrip public expectations. "I don’t think anyone is prepared for just how rapid progress will be from here," he cautioned.
Meanwhile, questions remain about Anthropic’s next moves:
- Delays in Flagship Model: Anthropic’s Opus 3.5 model, announced as a competitor to OpenAI’s o-series and Google’s Gemini Flash Thinking, is currently on hold. Reports suggest the performance gains didn’t justify the costs.
- Sonnet 3.5 Success: Despite setbacks, Opus 3.5 contributed to training Sonnet 3.5, which has since become the market’s most popular language model.
Jack Clark’s predictions underscore a pivotal moment in AI development. While computational challenges loom, the combination of innovative scaling techniques and traditional methods is expected to drive unprecedented breakthroughs in 2025. As competitors like OpenAI and Google push the boundaries of AI capabilities, Anthropic’s response will shape its role in this rapidly evolving landscape.