Cerebras has introduced the latest chip for artificial intelligence, the WSE-3, which is said to be twice as productive as existing solutions.
The WSE-3 chip, short for WAFER-Scale Engine, marks the third generation of the WAFER-Scale processor from Cerebras, boasting a size of 46.225 mm². With 4 trillion transistors manufactured using the TSMC 5-nanometer process, this chip contains 900,000 cores and 44 GB SRAM, delivering a performance of 125 AI petaflops, particularly notable for its high FP16 sparsity.
The new platform Cerebras CS-3, built around the WSE-3 chip, claims to offer double the productivity while maintaining the same energy consumption of 23 kW, a significant progression in line with Moore’s Law. Compared to NVIDIA’s H100, the WSE-3 chip provides approximately 57 times more and 62 times greater performance with enhanced FP16. However, when pitted against two DGX systems containing 16 H100 chips, the CS-3 system is only about 4 times faster when considering rarefied FP16 performance.
One of Cerebras’ key advantages is its memory capacity of up to 21 PB/s due to the 44 GB of built-in SRAM, a stark contrast to NVIDIA’s H100 with a maximum of 3.9 TB/s.
Regarding the CS-3 system, Cerebras intends to deploy its new systems in the AI Condor Galaxy 3 supercomputer, planned to span 9 data centers globally. The AI Condor Galaxy 3 cluster will be located in Dallas, Texas, and will leverage the new CS-3 platform to enhance overall computing performance to 8 AI exaflops. With all 9 centers utilizing 64 CS-3 platforms each, the total computational power will reach 64 AI exaflops, surpassing the initial estimate of 36 exaflops.
The company envisions CG-3 scaling to accommodate up to 2048 systems, capable of achieving a staggering 256 AI exaflops. Cerebras claims that such a system could train the LLAMA 70B model in just a day.
In addition to its next-generation accelerators, Cerebras has announced a collaboration with Qualcomm to