Meet Smaug-72B: The New King of Open-Source AI

A groundbreaking open-source language model has recently ascended to the top of the rankings on Hugging Face, a renowned platform for natural language processing (NLP) research. This model, named “Smaug-72B,” has quickly gained attention after being released by the startup Abacus AI. Known for helping businesses solve complex challenges in artificial intelligence (AI) and machine learning, Abacus AI’s latest model is a fine-tuned version of “Qwen-72B,” a model introduced by Alibaba Group just months ago.

What makes Smaug-72B so remarkable is its superior performance compared to well-established proprietary models such as GPT-3.5 by OpenAI and Mistral Medium by Mistral. In various widely recognized benchmarks, Smaug-72B has demonstrated stronger results, even surpassing the model it was derived from, Qwen-72B, by a considerable margin.

According to the Hugging Face Open LLM leaderboard, Smaug-72B is now the first and only open-source language model to achieve an average score above 80 across all major evaluations of language models. Although it hasn’t yet reached the 90-100 point range, which is typically associated with human-level performance, Smaug-72B’s emergence suggests that open-source models are beginning to compete with those developed by major technology companies, which have traditionally kept their advancements under wraps.

The release of Smaug-72B could signal a turning point in the AI landscape, empowering independent developers and organizations outside of tech giants to contribute to the evolution of artificial intelligence.

The Open-Source Edge

“Smaug-72B from Abacus AI is now available on Hugging Face, leading the LLM leaderboard as the first model to hit an average score of 80! Simply put, it’s the world’s best open-source foundation model,” stated Bindu Reddy, CEO of Abacus AI, in a post on X.com. Reddy also highlighted the company’s future plans, including the publication of their techniques in a research paper and the application of these methods to other advanced models, such as Mistral’s miqu, a fine-tuned version of LLama-2 with 70 billion parameters.

Reddy explained that the secret to Smaug-72B’s success lies in its focus on enhancing reasoning and mathematical abilities, as reflected in its high scores on GSM8K, a prominent math benchmark for language models. The forthcoming research paper will offer more insights into the methods used during the fine-tuning process, which specifically addressed common weaknesses in large language models, significantly boosting their performance in challenging tasks.

With today’s release, Smaug-72B has become the first open-source model to reach an average score of 80 on the Hugging Face Open LLM leaderboard, a significant milestone in the fields of natural language processing and open-source AI development. This accomplishment marks a new era of possibilities for AI researchers and developers working within the open-source ecosystem.

Smaug-72B’s standout performance in reasoning and mathematical tasks is largely attributed to Abacus AI’s unique approach to fine-tuning. These techniques, which will soon be shared in detail, have targeted the areas where many large models struggle, elevating Smaug-72B’s capabilities to new heights.

The Rise of Open-Source Models

Smaug-72B isn’t the only open-source model making waves. Qwen, the research team behind the original Qwen-72B, recently introduced Qwen 1.5, a series of smaller yet powerful language models ranging from 0.5 billion to 72 billion parameters. Qwen 1.5 models outshine several well-known proprietary models, including Mistral-Medium and GPT-3.5, and offer features like a 32k context length for improved performance across various applications.

Qwen also released Qwen-VL-Max, a cutting-edge vision-language model that challenges the dominance of Google’s Gemini Ultra and OpenAI’s GPT-4V, two of the most advanced proprietary vision-language models on the market today.

As Smaug-72B and other open-source models continue to close the gap with proprietary systems, the future of AI development is becoming increasingly open, collaborative, and innovative. With models like Smaug-72B leading the charge, the AI community is poised to accelerate progress at a pace and scale previously reserved for industry giants