Viewing profile — dhruvdh

Try `uvx pocket-tts serve`

El Capitan can also do FP8. HPC requires double precision generally but people are trying to make low precision work.

To be fair, you can buy ~3 of these for the price Nvidia charges for 24GB/32GB models.

To add, AMD only makes _parts_ of an MI300X server. It's like asking a tire manufacturer to give you a car for free.

I wish more people would just try to do things just like this and blog about their failures. > The published version of a proof is always condensed. And even if you take all the ma…

Disappointed that there wasn’t anything on inference performance in the article at all. That’s what the major customers have announced they use it for.

Which algorithm you pick for what shape of matrices is different and not straightforward to figure out. AMD currently wants you to “tune” ops and likely search for the right algori…

> despite them being fabless That's not how it works. You need to pump money into fabs to get them working, and Intel doesn't have money. If AMD had fabs to light up their money, t…

> Performance per watt was better for Intel No, not its not even close. AMD is miles ahead. This is a Phoronix review for Turin (current generation): https://www.phoronix.com/revie…

Is AMD behind hyperscaler in-house efforts? Outside of Google I don't think so.

Oh, maybe also change the title? I flagged it because of the title/url not matching.

I don't think having a common ancestry for the ISA means much, or even having the same ISA. Anyway, I don't understand what you want from me or are arguing about. They were trying …

Those are Vega, not CDNA. It wouldn't surprise me if those are rebranded consumer chips, though I haven't checked.

And yet Meta is using MI300X exclusively for all live inference on Llama 405B. Clearly there are workloads AMD wins at, and just going Nvidia by default for everything without cons…

You know AMD primarily sells CPUs right? For datacenter GPUs, they're going from ~500M-750M in 2023 full year (can't find proper numbers), to 4.5B+ full year 2024. In GPUs, it's al…

Batching is how you get ~350 tokens/sec on Qwen 14b on vLLM (7900XTX). By running 15 requests at once. Also, there is a Dockerfile.rocm at the root of vLLM's repo. How is it a pain…

Why would you use this over vLLM?

What's the point of the 8000 LOC limit? Has anyone worked in a project with a LOC limit? Why was the limit in place?

The MacBook NPU is 3x slower than the 45 TOPS threshold required for Copilot+ PC branding.

Yeah, and any updates to the model that make Recall attractive, cannot go back in time and reanalyze the past to be more useful. At least they will learn a lot from this, in 6-12 m…

The NPU runs this Silica model at 1.5 watts. MacBooks cannot even drive multiple monitors in this price range.

The FPGA being used is I believe one of the lowest speced SKUs. AWS instance prices are more of a supply/demand/availability thing, it would be more interesting to compare from a t…