Nearly two weeks after Elon Musk’s xAI startup opened up the AI model behind Grok to the general public, its AI chatbot is ready to get an upgrade.
The corporate announced Grok-1.5 on Thursday and claimed that its latest model can understand longer documents, handle more complex prompts, and perform more advanced reasoning.
While Grok-1.5 appears to be a step up from the unique 1.0 with improvements in coding and math skills, its announcement post shows that it still lags behind Google’s Gemini Pro 1.5 AI, OpenAI’s GPT-4, and Anthropic’s Claude 3 Opus in some benchmark tests, while outperforming OpenAI on one key HumanEval test.
Related: Meet Grok: Elon Musk Unveils ‘Spicy’ AI Chatbot Riddled With ‘Sarcasm’ and ‘Humor’
Grok-1.5 scored higher than GPT-4 on the HumanEval benchmark, which consists of 164 difficult programming problems not included within the AI model’s training data. GPT-4 had a rating of 67% and Gemini Pro 1.5 scored 71.9%, while Grok-1.5 received 74.1%.
Elon Musk’s xAI company is ready to release a new edition of the Grok AI chatbot, a ChatGPT competitor. Photo by Jaap Arriens/NurPhoto via Getty Images.
With a rating of 81.3% on the MMLU test, which covers knowledge of 57 subjects from an elementary to a sophisticated level, Grok-1.5 performed near Google Gemini’s rating (83.7%).
It also scored near GPT-4’s rating of 52.9% with a rating of fifty.6% on the MATH test, a benchmark that covers grade school to highschool math competition problems.
Related: Elon Musk Sues ChatGPT-Maker OpenAI, Accuses the Company of Working to ‘Maximize Profits For Microsoft, Quite Than For the Good thing about Humanity’
Musk stated in a Friday social media post that Grok 1.5 ought to be available on X, formerly Twitter, by next week.
The X owner has high expectations for the following generation of Grok, writing that the following step after Grok-1.5 will outperform the AI currently available “on all metrics.” Grok 2 is “in training now,” he wrote within the post.
Grok AI is currently only available to those with a $16 a month or higher Premium+ subscription on X.
Musk sued OpenAI, a competitor of xAI, earlier this month and asked for a court ruling that will force OpenAI to make the research and technology behind its AI public.