“Heuristic” AI models are becoming a trend, for better or worse

We call it the Logical Renaissance.

in On the heels of OpenAI’s o1 release,So-called Model Inference,There has been an explosion of inference models from competing,AI labs. In early November, DeepSeek, an AI research company funded by quantitative traders, released a preview of its first logical algorithm, Deep Sec-R1. In the same month, the Alibaba Coin team unveil What it claims is o1’s first “open” competitor.

So what opened the floodgates? Well, first, looking for new ways to improve generative AI technology. As my colleague Max Ziff recently did I mentionedHowever, “brute force” model scaling techniques no longer achieve the improvements they once did.

There is intense competitive pressure on AI companies to maintain their current pace of innovation. According to According to one estimate, the global AI market reached $196.63 billion in 2023, and could reach $1.81 trillion by 2030.

OpenAI, for example, has claimed that inference models can “solve harder problems” than previous models and represents a step change in the development of generative AI. But not everyone is convinced that inference models are the best way forward.

Amit Talwalkar, Associate Professor of Machine Learning at Carnegie Mellon says he finds the initial set of inference models “very impressive.” At the same time, however, he told me that he would “question the motives” of anyone who claims with certainty to know how far logic models will take the industry.

“AI companies have financial incentives to make rosy predictions about the capabilities of future versions of their technology,” Talwalkar said. “We risk focusing myopically on one model – which is why it is important for the broader AI research community to avoid blindly believing the hype and marketing efforts of these companies and instead focus on tangible results.”

There are two downsides to inference models, which are (1) they are expensive and (2) they are power-hungry.

For example, in the OpenAI API, the company charges $15 for every ~750,000 words o1 it analyzes and $60 for every ~750,000 words the model generates. This is between 3x and 4x the cost of OpenAI’s latest “irrational” model, GPT-4o.

O1 is available on OpenAI’s AI-powered chatbot platform, ChatGPTfree – with limits. But earlier this month, OpenAI was launched foot The O1 level is more advanced, the O1 Pro mode, which costs $2,400 per year.

“Total cost [large language model] “Logic is certainly not going away,” Jay van den Broek, a computer science professor at the University of California, told TechCrunch.

One reason why inference models are so expensive is that they require a lot of computing resources to run. Unlike most AI systems, O1 and other inference models try to verify their work as they do so. This helps them avoid some Pitfalls This usually results in models getting bogged down, with the downside being that they often take longer to come up with solutions.

OpenAI envisions future thinking models to “think” for hours, days, or even weeks on end. The company admits that the costs of use will be higher, but the gains will be greater Superbatteries for new cancer drugs – It might be worth it.

The value proposition of current inference models is less clear. Costa Huang, a researcher and machine learning engineer at the nonprofit Ai2, points out that o1 Not a very reliable calculator. A quick search on social media shows a number From o1 pro mode mistakes.

“These inference models are specialized and can perform poorly in general domains,” Huang told TechCrunch. “Some constraints will be overcome faster than others.”

Van den Broek asserts that inference models do not work actual thinking and are therefore limited to the types of tasks they can handle successfully. “True reasoning solves all problems, not just potential ones [in a model’s training data]He said. “This is the main challenge that still needs to be overcome.”

Given the strong market incentive to enhance inference models, they are sure to improve over time. After all, it’s not just OpenAI, DeepSeek, and Alibaba that are investing in this newest line of AI research. Venture capitalists and founders in adjacent industries are Merger About the idea of ​​a future dominated by artificial intelligence.

However, Talwalkar worries that large labs will sustain these improvements.

“It is understandable that large laboratories have competitive reasons to remain secretive, but this lack of transparency severely hampers the research community’s ability to engage with these ideas,” he said. “As more people work in this direction, I expect so [reasoning models to] Fast forward. But while some ideas will come from academia, given the financial incentives here, I expect most – if not all – of the models will be submitted by large industrial labs like OpenAI.