Unisami AI News

DeepSeek claims its ‘reasoning’ model beats OpenAI’s o1 on certain benchmarks

January 21, 2025 | by AI

pexels-photo-19926808

Introduction to DeepSeek-R1: A New Player in AI Reasoning Models

In the ever-evolving landscape of artificial intelligence, Chinese AI lab DeepSeek has made a remarkable stride with the release of an open version of its reasoning model, DeepSeek-R1. This model is touted to perform comparably to OpenAI’s o1 on specific AI benchmarks. What’s exciting is that R1 is readily available on the AI development platform Hugging Face under an MIT license, allowing its commercial use without any restrictions.

Benchmark Performance and Capabilities

DeepSeek claims that R1 outshines o1 in benchmarks such as AIME, MATH-500, and SWE-bench Verified. These benchmarks serve varied purposes; AIME evaluates model performance using other models, MATH-500 provides a set of word problems, while SWE-bench Verified focuses on programming tasks. What sets R1 apart as a reasoning model is its self-fact-checking ability, helping it avoid common pitfalls that often trap other models. Although reasoning models take slightly longer — seconds to minutes more — to derive solutions compared to non-reasoning models, they offer heightened reliability in fields like physics, science, and math.

The Technical Edge: Parameters and Accessibility

R1 boasts an impressive 671 billion parameters, as revealed in a technical report by DeepSeek. In the realm of AI, parameters are akin to problem-solving skills; models with more parameters generally exhibit superior performance. While 671 billion is massive, DeepSeek has also released “distilled” versions of R1 ranging from 1.5 billion to 70 billion parameters. Interestingly, the smallest version can even run on a laptop! The full-scale R1 demands more robust hardware but is accessible through DeepSeek’s API at prices significantly lower than OpenAI’s o1 by 90%-95%.

Challenges and Controversies

However, there’s a caveat to using R1: being a Chinese model means it must adhere to China’s internet regulations ensuring responses align with “core socialist values.” Consequently, R1 avoids answering questions about sensitive topics like Tiananmen Square or Taiwan’s autonomy. Many Chinese AI systems, including reasoning models, sidestep discussions that could attract regulatory scrutiny within the country.

“The impressive performance of DeepSeek’s distilled models […] means that very capable reasoners will continue to proliferate widely and be runnable on local hardware,” said Dean Ball, an AI researcher at George Mason University.

{Dean Ball}

Geopolitical Impacts and Future Prospects

The release of R1 comes amidst geopolitical tensions, particularly following the Biden administration’s proposal for stricter export rules on AI technologies for Chinese companies. Currently, firms in China face limitations on purchasing advanced AI chips. If new rules are enforced as intended, they could impose tighter restrictions on both semiconductor technology and models essential for developing sophisticated AI systems.

In response to these developments, OpenAI has urged the U.S. government to bolster domestic AI advancements to prevent being outpaced by Chinese models. OpenAI’s VP of policy Chris Lehane specifically mentioned High Flyer Capital Management, DeepSeek’s parent company, as a significant entity of concern.

Conclusion: The Road Ahead for Chinese AI Labs

As it stands, at least three Chinese labs — DeepSeek, Alibaba, and Kimi (owned by Moonshot AI) — have introduced models they claim rival OpenAI’s o1. Notably, DeepSeek was the pioneer with its preview announcement of R1 last November. This progression suggests that Chinese AI labs will continue as “fast followers,” rapidly advancing their capabilities while navigating the challenges posed by regulatory environments and international policies.

Image Credit: Jules Clark on Pexels

RELATED POSTS

View all

view all