A 10 year old Xeon is all you need

Post by ethbr1 » Mon, Jun 01, 2026, 2:06 PM UTC

fhars wrote: post #87

cafkafk wrote: post #6

Earlier quoted context omitted.

> (purple on black is really hard to read) Noted, and agree (it looks like it has also already been clicked, which I dislike). I honestly I need to redo the themes. > You say it runs "at reading speed". Have you benchmarked it? At some point a few weeks ago, yes I think so, but I didn't write it down for some reason... so I'll have to find a time when it's not busy and do it again without a noisy system. Right now th…

And if you ever run out of things to do in your copious free time, it looks like that PR #1744 was merged without the has_target_ctx assert two days after you uploaded your drafter quants. So you can now redo all your quants and rerun all your benchmarks ;-).

Post by souterrain » Mon, Jun 01, 2026, 2:11 PM UTC

dangus wrote: post #129

RetroTechie wrote: post #106

Earlier quoted context omitted.

How many kWh to fabricate a brand new machine better suited to the task? As long as performance is useable (apply your own metrics!), pulling it from existing hardware is likely the option with the lower eco footprint. Also: chances are it'll only be used for this purpose occasionally, and/or for a short while. In that scenario [fabricating new hardware] always has the bigger eco footprint.

I don’t know why you’d assume that an older system is lower footprint. If you’ve got something consuming 100 watts average over your 24 hour period, and your electricity costs 20 cents per kWh, you’re already spending almost as much as a Claude subscription. Just on electricity, this assumes your hardware never fails and you never incur any additional costs. There’s a big reason why newer more efficient hardware is i…

Post by bee_rider » Mon, Jun 01, 2026, 2:18 PM UTC

qingcharles wrote: post #172

Would there be any advantage of running this as dual Xeon? The CPUs are $5 and a dual mobo is $50...

Post by selicos » Mon, Jun 01, 2026, 2:27 PM UTC

ryandvm wrote: post #123

cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

I disagree. We are currently in a weird period where these frontier AI companies are losing tons of money even on the subscription-based AI models. It's just too compute intensive and there's no way most people are going to be buying the kind of hardware required to run $20 worth of inference every day. Sadly - it's going to be ads. Advertising is going to get in there and enshittify the whole thing because as always…

Post by Lerc » Mon, Jun 01, 2026, 2:28 PM UTC

cafkafk wrote: post #2

Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma 4) running at reading speed on an old recycled server with a single Xeon E5-2620 v4 and 128GB of DDR3 RAM (and no GPU). It took a lot of work, but it actually worked out somehow.…

    CPU(s): 32
      On-line CPU(s) list: 0-31
    Vendor ID: GenuineIntel  
    Model name: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz