Meta MTIA v2 – Meta Training and Inference Accelerator
1–10 of 63 posts
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#2Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#3Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.
They are different architectures optimized for different things.
From the Meta post: "This chip’s architecture is fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity for serving ranking and recommendation models."
Optimizing for ranking/recommendation models is very different from general purpose training/inference.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#4Pretty large increase in performance over v1, particularly in sparse workloads.
Low power 25W
Could use higher bandwidth memory if their workloads were more than recommendation engines.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#5My mind still boggles that a BBS+ads company would think it needs to design its own chips.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#6My mind still boggles that a BBS+ads company would think it needs to design its own chips.
"Depending on how you want to think about it, it was funny or inevitable or symbolic that the robotic takeover did not start at MIT, NASA, Microsoft or Ford. It started at a Burger-G restaurant ..."
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#7Pretty large increase in performance over v1, particularly in sparse workloads. Low power 25W Could use higher bandwidth memory if their workloads were more than recommendation engines.
First gen was 25W. The new one is 90W.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#8My mind still boggles that a BBS+ads company would think it needs to design its own chips.
Well, the first commercial computer was created by a company whose primary business was running cafes... https://en.wikipedia.org/wiki/LEO_(computer)
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#9Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.
They are different architectures optimized for different things. From the Meta post: "This chip’s architecture is fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity for serving ranking and recommendation models." Optimizing for ranking/recommendation models is very different from general purpose training/inference.
Yeah, it may fit their current workload perfectly, but it doesn't seem very future proof with the limited bandwidth. Given how fast ML is evolving these days I question if it makes sense to design and deploy a chip like this. I guess they do have a very large workload that will benefit immediately.
Re: Meta MTIA v2 – Meta Training and Inference Accelerator
#10My mind still boggles that a BBS+ads company would think it needs to design its own chips.
"Depending on how you want to think about it, it was funny or inevitable or symbolic that the robotic takeover did not start at MIT, NASA, Microsoft or Ford. It started at a Burger-G restaurant ..." https://marshallbrain.com/manna1
dangit, I've got things I should be doing. Posting interesting stories during business hours continues grumbling incoherently