In-depth Look at Tensor Flow Processor (TPU) for Machine Learning and AI
"Discover the power of Tensor Flow Processor (TPU) in optimizing machine learning and AI operations, as Google revolutionizes faster processing. Learn about its deployment, capabilities, and efficiency compared to traditional CPUs and GPUs."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Tensor Flow Processor (TPU) 1 B. RAMAMURTHY 5/13/2025 CSE541, B. Ramamurthy
References 2 https://developers.google.com/appengine/ Rick Merrit, EE|Times: Google Revs Faster TPU: http://www.eetimes.com/document.asp?doc_id=1331753&_mc=RSS_ EET_EDT. 5/17/2017. 5/13/2025 CSE541, B. Ramamurthy
A little bit of History 3 Technology runs in cycles --David Patterson In his famous paper Reduced Instruction Set Architecture by David A. Patterson, CACM, Jan 1985, vol 28, number 1. That time it was CISC RISC Now it is RISC ASIC (CISC) and who is behind it? 5/13/2025 cse541, B. Ramamurthy
Overview 4 Tensor flow processor is an ASIC (Application Specific Integrated Circuit) that is optimized for machine learning (Ml) and artificial intelligence (AI) algorithms and operations. Four of these TPUs powering a cloud service can deliver 180 Tflops for ML tasks. Set of the TPU is the engine / computation power behind the TPU-based tensor flow cloud services for (i) training (data) and testing and (ii) inference tasks. 5/13/2025 cse541, B. Ramamurthy
Introduction 5 How to use the TPU? Google provides services to use the TPU for research and commercial purposes. Think of all the data that is collected on the cloud? Hmm.. What can you do with it? Can use it for instant inference and realtime response to events if the analysis is powered by powerful high speed processors. That is the general idea of TPU. The effort is to harness the rising interest in ML to drive use of Google s cloud services. While the TPU design itself shrouded in secrecy, TensorFlow framework the software interface to the chip is open-source. The Cloud TPU as it is know supports floating-point math which Google encourages for both training and inference phases of ML. 5/13/2025 cse541, B. Ramamurthy
Actual Deployment 6 Google is packing 4 TPU on a custom accelerator board and 64 of these boards are packed on a two dimensional torus network in a cluster called a pod that is capable of up to 11.5 petaflops. Of course, lots of heat sinks How about performance? 8 TPUs completed in 6 hours what 32 high GPU took a full day to complete (Training task). In 2015 using Google s Ml jobs, this TPU ran 15-30 times faster and delivered 30-80 times better performance per watt than Intel s Haswell server CPU and Nvidia s K80 GPU. 5/13/2025 cse541, B. Ramamurthy
What is inside the Cloud TPU? 7 Of course, Google provide very little details of what inside the ASIC and the support systems. Earlier version offered only inference: so was supporting only quatized integer computation. Latest version supports training and so requires floating point operations. Super-fast memory bandwidth, significant on-chip memory,. Things are happening on the board and not on the rack. The Cloud TPU supports one large, one medium and 8 small heat sinks. As for the instructions: linear algebra instructions, core matrix multiple instructions. 5/13/2025 cse541, B. Ramamurthy
Programming 8 Commercial users can develop applications using a new alpha program on Google Cloud Platform. Can have free access to 1000 Cloud TPUs: Tensor Flow Research cloud Go find applications, may be many in driverless car initiatives that require instant inferences/responses with hard deadlines? Most of the applications are the ones that require the so called deep learning and often involve neural networks. 5/13/2025 cse541, B. Ramamurthy
How do you get started? 9 See Google pages on getting started. https://www.tensorflow.org/ https://www.tensorflow.org/get_started/get_started https://www.tensorflow.org/tfrc/ 5/13/2025 cse541, B. Ramamurthy
Tensor flow Docker Image 10 Lets demo a docker image of tensorflow on Jupyter. Stack: Docker bundling Jupyter with Tensorflow kernel Handout 5/13/2025 cse541, B. Ramamurthy
