MTIA v1: Meta’s first-generation AI inference accelerator
ai.facebook.com
MTIA v1: Meta’s first-generation AI inference accelerator
1–10 of 50 posts
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#2How do they compare to TPUs?
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#3Why does the headline just mention inference when the acronym also mentions training?
Is it primarily for inference and the training is just an after thought?
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#4>>>> fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W.
So 2 generation of immediate improvement available.
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#5>>>> fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W. So 2 generation of immediate improvement available.
Their goal is to beat GPUs
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#6Re: MTIA v1: Meta’s first-generation AI inference accelerator
#7>>>> fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W. So 2 generation of immediate improvement available.
TOPS might take a more complex operation as a unit, same as computing shader passes per second might mean a very simple computation or a very complex operation every s^-1
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#8Comparing MTIA v1 vs Google Cloud TPU v4:
MTIA v1's specs: The accelerator is fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W. Up to 128 GB of ram LPDDR5.
Googles Cloud TPU v4: 275 teraflops (bf16 or int8), 90/170/192 W. 32 GiB of HBM2 RAM, 1200 GBps. From here: https://cloud.google.com/tpu/docs/system-architecture-tpu-vm...
So it seems that the Google Cloud TPU v4 has an advantage in terms of compute per chip and ram speed, but the Meta one is much more efficient (2x to 4x, it is hard to tell) and has more ram but it is slower ram?
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#9Why does the headline just mention inference when the acronym also mentions training? Is it primarily for inference and the training is just an after thought?
It looks rather general-purpose (for ML tasks) to me:
Each PE is equipped with two processor cores (one of them equipped with the vector extension) and a number of fixed-function units that are optimized for performing critical operations, such as matrix multiplication, accumulation, data movement, and nonlinear function calculation. The processor cores are based on the RISC-V open instruction set architecture (ISA) and are heavily customized to perform necessary compute and control tasks.
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#10Why does the headline just mention inference when the acronym also mentions training? Is it primarily for inference and the training is just an after thought?
These seem power and density optimized. This sort of custom hardware is all about supply chains and getting a lot of them everywhere. This flavors the inference use-case.
For large training jobs it is more about turn around time; running hideously expensive GPUs sucking down huge amounts of power is fine.