Groq large model inference chip has excellent performance and consumes only one-tenth of NVIDIA GPU

Groq's large model inference chip proposed the LPU (Language Processing Unit) solution for the first time in the world. This is a Tensor Streaming Processor (TSP) chip based on the TSA architecture. Although Groq's LPU does not use expensive cutting-edge process technology, but chooses the 14nm process, its performance is still excellent, providing up to 1000 TOPS of computing power, and its performance on some machine learning models is 10 to 100 times higher than that of conventional GPUs and TPUs. In addition, the energy consumption of the Groq LPU chip is only one-tenth of that of NVIDIA GPUs.