Amazon cashing in on the surge of interest in generative AI

Graviton4 provides up to 30 per cent better compute performance, 50 per cent more cores, and 75 per cent more memory bandwidth than current generation Graviton3 processors, delivering the best price performance and energy efficiency for a broad range of workloads running on Amazon EC2

Designing our own chips is the best path to make our customers’ lives better and easier in the generative AI era, Amazon CEO Andy Jassy said.

Jassy said that several years ago, when Amazon started pursuing building its own chips, “a lot of folks thought this was nuts”.

“We heard a lot of the same refrains you often hear — why make this investment, why invest in a team and all the other fixed costs to develop your own chip when you can buy from other suppliers?” Jassy posted on X during their ‘re: Invent 2023’ conference here.

He said that Amazon was lucky to find and join forces with the amazing Annapurna Labs team, who started with a chip (named Nitro) that offloaded security, networking, and some other virtualization functions from our servers so customers could use more of the server than they could before.

“Then, that team built a generalised CPU chip, Graviton, which has been very popular and impactful for customers, before embarking on building custom AI chips — Trainium (for training) and Inferentia (for inference)—which are also off to a strong start,” Jassy informed.

Earlier this week, AWS CEO Adam Selipsky unveiled the next generation of two AWS-designed chips — Graviton4 and Trainium2 — for a broad range of customer workloads, including machine learning (ML) training and generative artificial intelligence (AI) applications.

Graviton4 provides up to 30 per cent better compute performance, 50 per cent more cores, and 75 per cent more memory bandwidth than current generation Graviton3 processors, delivering the best price performance and energy efficiency for a broad range of workloads running on Amazon EC2.

Jassy said he is very excited about the most recent chips.

“Graviton4 marks the fourth generation we’ve delivered in just five years, and it’s the most powerful and energy efficient multipurpose chip we have built to date,” Amazon CEO said.

With the surge of interest in generative AI, Trainium2 will help customers train their ML models faster, at a more advantaged price-performance, he added.