A US startup desires to problem Nvidia’s maintain over the AI GPU market by harnessing its personal chips to run AI workloads sooner and extra cheaply. The startup is named Cerebras Methods, which has been growing iPad-sized processors known as the Wafer Scale Engine. On Tuesday, it introduced a service that’ll let clients use its chips for their very own AI packages. The primary distinction is that Cerebras claims its expertise can run generative AI packages as much as 20 instances sooner however at one-fifth the price of Nvidia GPUs, such because the H100, an AI processor extensively used within the trade. Referred to as Cerebras Inference, the service refers to an AI program’s skill to generate new information, like predicting the following phrase whereas writing a chunk of textual content. Cerebras says its personal platform is the “quickest AI inference answer on the earth.” To display this, the corporate is utilizing Wafer Scale Engine chips to energy Meta’s Llama 3.1 open-source giant language mannequin. The outcomes could make Llama 3.1 present solutions with seemingly no lag.
This Tweet is presently unavailable. It may be loading or has been eliminated.
Particularly, Cerebras says its chips can run the 8 billion parameter model of Llama 3.1 to churn out 1,800 tokens per second, with every token representing about 4 characters within the English language. In different phrases, the AI program can produce a 1,300-word article in a single second.The identical chips can even run the extra highly effective 70 billion parameter model of Llama 3.1 to provide 450 tokens per second. In each benchmarks, Cerebras claims its expertise far exceeds the token-per-second efficiency of AI cloud suppliers, together with Amazon’s AWS, Microsoft Azure, and Groq.
(Credit score: Cerebras Methods)
Constructing chatbot providers round Cerebra’s implementation of Llama 3.1 additionally guarantees to be low-cost, in line with the startup. “Cerebras Inference is priced at a fraction of GPU-based rivals, with pay-as-you-go pricing of 10 cents per million tokens for Llama 3.1 8B and 60 cents per million tokens for Llama 3.1 70B.” In distinction, ChatGPT developer OpenAI—an Nvidia chip consumer—can cost third-party firms $2.50 to $15 per a million tokens. Cerebras has been constructing its Wafer Scale Engine chips utilizing Taiwan’s TSMC, the contract chip producer behind Nvidia’s personal AI GPUs. In March, Cerebras launched the third-generation chip, the WSE-3, which boasts 4 trillion transistors, together with 900,000 AI cores.
Beneficial by Our Editors
(Credit score: Cerebras Methods)
In a weblog put up, the startup additionally took some photographs at Nvidia by saying the corporate’s WSE-3 presents 7,000 instances extra reminiscence bandwidth over the H100. “Cerebras solves the reminiscence bandwidth bottleneck by constructing the biggest chip on the earth and storing your complete mannequin on-chip. A Wafer Scale Engine has 44GB of on-chip SRAM,” the startup added in a tweet. Nonetheless, Cerebras is benchmarking the WSE-3 towards older Nvidia expertise, contemplating the H100 was first launched in 2022. In March, Nvidia unveiled the successor within the Blackwell structure, which guarantees to supply a seven- to 30-times efficiency enchancment over the H100. The corporate plans on transport Blackwell by means of new GB200 and B200 merchandise later this 12 months, though The Info reviews the GPUs might be delayed by three months.Nvidia did not instantly reply to a request for remark. Within the meantime, Cerebras is eyeing increasing entry to its WSE-3 chip. This contains making the silicon obtainable to different cloud suppliers. However in no shock, the corporate’s AI chip is not low-cost. Cerebras advised PCMag it sells the WSE-3 by means of its CS-3 {hardware} providing, which prices “a pair million per system.” That is considerably costlier than a single H100 GPU, which may go for round $30,000.
Get Our Finest Tales!
Join What’s New Now to get our high tales delivered to your inbox each morning.
This text might include promoting, offers, or affiliate hyperlinks. Subscribing to a publication signifies your consent to our Phrases of Use and Privateness Coverage. You might unsubscribe from the newsletters at any time.
About Michael Kan
Senior Reporter
I have been with PCMag since October 2017, masking a variety of subjects, together with client electronics, cybersecurity, social media, networking, and gaming. Previous to working at PCMag, I used to be a overseas correspondent in Beijing for over 5 years, masking the tech scene in Asia.
Learn Michael’s full bio
Learn the newest from Michael Kan
