Viewing profile — joefourier

What about multi-token prediction and speculative diffusion? That’s a different mechanism of prediction, even if it serves only to accelerate decoding.

You cannot just "try all possible optimisations". It takes time, effort, and money that could otherwise be spent elsewhere (especially for training, where each training run is espe…

Anthropic's API has two nines availability and Claude Code is a TUI made with React that can regularly consume more than 1GB of RAM, and the codebase is utter slop. They couldn't f…

> Like isn't it weird that the 1 million parameter model with the same architecture can't solve basic puzzles but suddenly the 1 trillion parameter can conjure up counter-examples …

I stopped using Cursor because of how terribly optimised it is (worse than VSCode despite being a fork). It would routinely take up 50% of the CPU resources on my MacBook M4 and gi…

Why is it unethical? I'm both a freelance engineer and a business owner that sells software, and I've both sold my labour for equity/revenue share, and for a flat hourly rate. If I…

> My response is “great! Let businesses take a lesson here: give all your employees a chunk of the company. Let’s all share in the success!” Don't >95% of tech companies offer stoc…

Yeah most of the performance increases have mostly been from architectural improvements like reduced precision tensor cores. AFAIK FP4 is basically the limit for floating point mat…

Demand is so high and supply so low customers will go to anyone that has any gear, period. Anthropic is paying xAI for GPUs from 2022, not the latest Nvidia release.

And yet Anthropic is paying xAI over a billion dollars a month for those out of date GPUs in their first datacentre (H100s being nearly 4 years old at this point). Even A100s are s…

Do you think the work will still apply to speculative/alternative decoding methods like MTP and block diffusion, which are making batch=1 decoding less memory bound? Kernel launch …

You'd be surprised, people are somehow buying Tesla P40s and M40s on eBay for almost $300 and $180 respectively (M40 being the same gen as GTX 950). Google Colab still offers T4s a…

Outside of training the biggest LLMs at big labs, GPU lifespan isn't as short as the OP made it out to sound. A100s are 6 years old and still a reliable work-horse, and the 80GB ve…

> And why is V100 even used? V100 is four generations old and not even supported anymore. It wouldn’t surprise me that due to bureaucratic processes, it’s still somehow the most re…

Not a single new 64GB GPU, but multiple used GPUs. They’ve significantly increased in price (so much for hardware depreciation…) but you can still get a modded 22GB 2080 ti for $32…

What quant? You should have no problem running it at Q4 with 256K context, Q5 or Q6 even although maybe not at full context. I can run Q4 on a 4090 with just 24GB VRAM.

Who is going to buy a $4299 M5 Max MBP with 64GB of RAM just to run Gemma 4 31b? Firstly you don't need 64GB for that model. Secondly if you want a machine that sits in the corner …

Why didn't you take into account batching, input tokens, different costs of electricity, and the fact that a laptop can still hold a decent % of its resale value, and is useful for…

> I'd compare it to OpenAI 5 years ago except I think even then OpenAI had way more! Say what? 5 years ago OpenAI had received around $139 million in funding, and they’d just come …

It's incredibly common all over Europe, not just Switzerland. Not only the metros but the trams and even buses often rely on this system where there's no turnstile or barrier, you …

Personally I feel like it would be less undignified and infantilising to have a machine take care of my basic bodily functions than a human being. There's no feeling of judgement o…

> I also used dedicated servers in the late ’90s (and they still offer great value today). But before AWS, provisioning new hardware typically took days, not minutes. VPSes and non…

> Cloud computing was an absolutely mind blowing revolution - suddenly your startup could run its own computer systems in minutes without need to install and run your own systems i…

> In every country, men commit almost all violent crimes. In school, boys physically bully other boys. Hence the physical punishment for them. As I've said, and @echoangle repeated…

Boys and girls being different does not mean one sex deserves corporal punishment and one does not. Girls are equally capable of cyberbullying (which is covered by this law), why s…