Viewing profile — mmastrac

I've been pondering a smaller quantization and experts swapping for this! Happy to take on PRs if you want to experiment as well.

I've been working on a fresh, AI assisted port of DiffusionGemma from scratch and it takes a significant amount of time to deslop. I've spend a nonzero amount of time on refactorin…

I suspect this means that there's just a ~10% inefficiency in token to information mapping.

Which one? Nemotron Diffusion? It's impossible to say for sure, but I have a fairly deep library of metal kernels that _might_ cover some of the nvidia model's architecture.

Awesome. I may need to finally bite the bullet and upgrade my macOS to test out the MPP approach you've taken. I've got a number of tiled-load kernels, and a top-k attention kernel…

TBH, I think there's some truth to that. I spent _ages_ tuning the kernels to match the tested FLOP count of my M3's processor. I only have an M3 though and wasn't able to push int…

I have a project that's almost ready to run DiffusionGemma as well. The two project might potentially work well together. I'm getting ~20tok/s on a 36GB M3 and there's strong possi…

The founding fathers would be aghast at what America became. Hard to imagine modern America passing some of those constitutional amendments that older America passed wayyy back.

Is it just me or does this post not mention how much RAM they had? I would love to know - I have a dual-Xeon 1U screamer with 96GB of DDR4 RDIMM just sitting around... FWIW I'm get…

There's a bot user that seems to crosspost every lobsters story here if it isn't already.

This sort of thing is a lot of fun. I've been going smaller.. I have a custom-quantized Rust port of DiffusionGemma (26B) that seems to perform better (in responses) than benchmark…

Author here - if anyone has any contacts at Cloudflare to get the proxied domains (at least roadpay[.]cc) taken down, that would be great. I wasn't able to get an abuse report to s…

They are long-lived if you're a mayfly. But I think the point is that they should be cheap to set up, and because of the short life, never really contain anything except the potent…

Om Malik was the guy who had the biggest influence on the direction of my life, by far. It was through him I met Naval Ravikant in 2007, and then through Naval I met my co-founder …

One of the most fascinating things to fall out of the AI apocalypse is seeing how abundant AI access amplifies the qualities of a company. IMO, Deno has always been more methodical…

The consequences of plugging the wrong voltage or polarity into a barrel jack, most of which are compatible enough, is pretty bad. Ask me how I know.

FWIW this post has both a "thanks" section for the human reviewers, and numerous footnotes linking to more authoritative sources.

I'd love it if WASI modules could introspect their own custom sections (potentially even more introspection than that), but I've never been able to figure out a good way to do this…

Author here, happy to answer any questions. I've been working on building some higher-level abstractions on link sections (specifically, link-time optimized collections like maps (…

Looks great, but why isn't there a checkbox to turn off the chunk of pork fat? I use avocado oil, personally.

Types and generics are hard to parse in regular macros without a tt muncher. Ditto for fn args. If you need to do actual matching of types, you can't capture them as $xxx:ty becaus…

I like the idea. I think I'd feel more comfortable if I could drop an .rs file into the eBPF dir instead of a .c one. It's already a Rust project! :) And for some reason I was expe…