Anthropic says its new Claude 3 AI chatbot outperforms GPT-4 on key benchmarks

The battle between AI chatbots is more than a two-horse race. Anthropic, the company founded by several former OpenAI employees, claims that its new Claude 3 language model outperforms Google’s ChatGPT and Gemini in several key industry benchmarks. It even reached “almost human” levels in some tasks, the company said blogged.

There are three new chatbots under the Claude 3 umbrella, including Haiku, Sonnet, and Opus. A sonnet feeds Claude.ai chatbot and is available for free with email login. Meanwhile, Opus is the largest and most powerful LLM and will be available with a $20 per month subscription through the “Claude Pro” service. It is also multimodal, so it can work with both text and images, unlike previous versions.

All Claude 3 models “can power live customer chats, auto-complete and data mining tasks where responses need to be immediate and in real-time,” the company said. In addition to promising “near-instant results,” they are supposed to be able to handle longer, multi-step instructions with increased accuracy.

Opus demonstrated better graduate-level reasoning than the GPT-4, scoring 14.7% higher on this test than the GPT-4. It also beat the OpenAI chatbot on tasks involving math, coding, reasoning and knowledge.

They are also on top of past Claude models. “For most workloads, Sonnet is 2x faster than Claude 2 and Claude 2.1 with higher levels of intelligence. It excels at tasks requiring quick responses, such as knowledge mining or sales automation. Opus delivers similar speeds to Claude 2 and 2.1, but with much higher levels of intelligence,” according to Anthropic.

Meanwhile, the Haiku, the smallest version of the Claude 3, is “the fastest and most cost-effective model on the market.” To that end, he is able to read a dense research paper full of charts and graphs in less than three seconds.

The company also noted that Claude 3 “can handle a wide range of visual formats, including photos, charts, graphs, and technical diagrams,” helping companies that use PDF files, flowcharts, or presentation slides. It will also be less likely to reject innocuous content thanks to a more nuanced understanding of requests while recognizing “true harm”.

Anthropic said that the Claude AI is guided by 10 secret foundational pillars of justice. Claude 3 was trained on both non-public internal and public data using hardware from Amazon Web Services (AWS) and Google Cloud (Amazon recently invested $4 billion in Anthropic).

Claude 3 Opus and Claude 3 Sonnet are now available through Anthropic’s API, with Haiku to follow soon. Sonnet is also available through Amazon Bedrock and in private preview at Google Cloud’s Vertex AI Model Garden.

This article contains affiliate links; if you click on such a link and make a purchase, we may earn a commission.