Undoubtedly, 2023 has emerged as a pivotal year for generative AI, and Google is concluding it with noteworthy advancements in artificial intelligence. The company has introduced its most potent Tensor Processing Unit (TPU), officially named Cloud TPU v5p, and an AI Hypercomputer as part of Google Cloud. Amin Vahdat, Google’s Engineering Fellow and Vice President for Machine Learning, Systems, and Cloud AI, remarked on the significant growth in generative AI models, witnessing a tenfold increase in parameters annually over the past five years.
Functioning as an AI accelerator for training and serving models, Cloud TPU v5p is specifically designed by Google to handle large models with extended training periods, predominantly composed of matrix computations and devoid of custom operations within the primary training loop, such as TensorFlow or JAX. Each TPU v5p pod incorporates 8,960 chips, leveraging Google’s highest-bandwidth inter-chip interconnect.
Building upon previous iterations like v5e and v4, the TPU v5p boasts twice the FLOPs and is four times more scalable in terms of FLOPS per pod compared to its predecessor, TPU v4. Notably, it can train LLM models 2.8 times faster and embed dense models 1.9 times faster than the TPU v4.
Next up is the AI Hypercomputer, featuring an integrated system encompassing open software, performance-optimized hardware, machine learning frameworks, and adaptable consumption models. The objective is to enhance productivity and efficiency by presenting these components as a unified solution rather than individual entities. The AI Hypercomputer’s hardware is fine-tuned for performance, leveraging Google’s Jupiter data center network technology.
In a departure from the norm, Google extends open software access to developers with “extensive support” for machine learning frameworks like JAX, PyTorch, and TensorFlow. This announcement coincides with Meta and IBM’s AI Alliance launch, emphasizing open sourcing, a realm in which Google notably does not participate. The AI Hypercomputer introduces two models, namely Flex Start Mode and Calendar Mode.
This announcement aligns with the introduction of Gemini, a novel AI model hailed by Google as its “largest and most capable.” The model is now available on Bard and the Pixel 8 Pro, offered in three sizes: Gemini Pro, Gemini Ultra, and Gemini Nano.
