Has there been any rumors or statements from Facebook on them eventually stepping into selling cloud compute? I'd be surprised if they are investing in building hardware accelerators just for their own services.
MTIA v1: Meta’s first-generation AI inference accelerator
11–20 of 50 posts
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#12Just missed FP8 implementation on hardware
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#13Has there been any rumors or statements from Facebook on them eventually stepping into selling cloud compute? I'd be surprised if they are investing in building hardware accelerators just for their own services.
Their footprint for just their own services rivals some other public clouds.
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#14It's curious why nobody is selling these systems yet
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#15Has there been any rumors or statements from Facebook on them eventually stepping into selling cloud compute? I'd be surprised if they are investing in building hardware accelerators just for their own services.
I think they’d be bad at it for the same reason google is bad it. Enterprise sales is not in their dna.
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#16Has there been any rumors or statements from Facebook on them eventually stepping into selling cloud compute? I'd be surprised if they are investing in building hardware accelerators just for their own services.
I think they’d be bad at it for the same reason google is bad it. Enterprise sales is not in their dna.
The AI inference/training market is so competitive that I doubt enterprise sales is going to be the problem. A company planning on spending $50M training a model is not going to be convinced by some smooth talking sales guy over a golf game. They will look at the actual price/performance.
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#17It's curious why nobody is selling these systems yet
Probably the software needs to be optimized for the hw and also the hw may not be general purpose enough even if offered. People demand nvidia because cuda is very optimized for their gpus and many AI software use cuda
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#18Comparing MTIA v1 vs Google Cloud TPU v4: MTIA v1's specs: The accelerator is fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W. Up to 128 GB of ram LPDDR5. Googles Cloud TPU v4: 275 teraflops (bf16 or int8), 90/170/192 W. 32 GiB of HBM2 RAM, 1200 GBps. From here: https://cloud.google.com/tpu/docs/s…
Is there something that compares these to more consumer offering like Apple’s ANE?
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#19Comparing MTIA v1 vs Google Cloud TPU v4: MTIA v1's specs: The accelerator is fabricated in TSMC 7nm process and runs at 800 MHz, providing 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision. It has a thermal design power (TDP) of 25 W. Up to 128 GB of ram LPDDR5. Googles Cloud TPU v4: 275 teraflops (bf16 or int8), 90/170/192 W. 32 GiB of HBM2 RAM, 1200 GBps. From here: https://cloud.google.com/tpu/docs/s…
FWIW, you're comparing a training-specialized chip to an inference-specialized chip. It'd be more apples to apples to compare to TPU v4 lite, but I can't find that chip's details anywhere beyond some mentions in the TPU v4 paper: https://arxiv.org/abs/2304.01433
Re: MTIA v1: Meta’s first-generation AI inference accelerator
#20Earlier quoted context omitted.
I think they’d be bad at it for the same reason google is bad it. Enterprise sales is not in their dna.
The AI inference/training market is so competitive that I doubt enterprise sales is going to be the problem. A company planning on spending $50M training a model is not going to be convinced by some smooth talking sales guy over a golf game. They will look at the actual price/performance.
You’d be surprised. Azure is growing like gangbusters on the backs of smooth talking salesmen taking CTOs golfing.