Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A 10 year old Xeon is all you need

point.free

Post reply on HN
Print view View original

121–130 of 301 posts

Re: A 10 year old Xeon is all you need

#121

Post by brightball »

benterix wrote: post #98
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

I wouldn't say "completely implode", too much money was poured int it, but it's clear we're heading in that direction. You get a model that is "good enough", plus privacy, plus savings in the long term. Paradoxically, the better results we get from general harness of coding agents, the less moat Claude and co. get. It's unbelievably how fast some open models outpaced frontier models of just a few months ago.

I keep intending to find time to try them. What are you seeing the best results with?

Re: A 10 year old Xeon is all you need

#123

Post by ryandvm »

cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

I disagree. We are currently in a weird period where these frontier AI companies are losing tons of money even on the subscription-based AI models. It's just too compute intensive and there's no way most people are going to be buying the kind of hardware required to run $20 worth of inference every day.

Sadly - it's going to be ads. Advertising is going to get in there and enshittify the whole thing because as always, advertising income is too easy and too plentiful for any company to resist.

Right now the models are fairly agnostic, but we are a hair-breadth away from ChatGPT responding with, "the right tool for this job is a circular saw - something like the Milwaulkee M18, which happens to be on sale at Home Depot this weekend."

Re: A 10 year old Xeon is all you need

#124

Post by BirAdam »

throwaway2037 wrote: post #65
jansommer wrote: post #52

The E5-2620 v4 is great. Have been using it for 10 years now. Wanted to upgrade until I saw current prices. I have 64 GB ddr4. Paired it with rx 9060 xt 16 GB and games run as fast as ever. Perhaps the cpu is a slight bottleneck in DOOM The Dark Ages, but i'm at 60 fps, so no problem. Light llm on the gpu is a nobrainer, and it's cool to see that things can be tuned to run ok on the cpu. I bought 2667 v4 a month ago…

> The E5-2620 v4 is great. Have been using it for 10 years now. 10 years? Damn, that is a long time. I always assumed that heat-induced damage will kill a CPU after a certain amount of time (5-7 years). Am I wrong here? I assume yes. Or are CPUs must stronger/tougher than the bad old days?

This is among the "real" differences between workstation/server CPUs and commodity chips for laptops/desktops/handhelds.

Even then, if a commodity chip isn't pushed full tilt at all times, and assuming that the venting and dissipation are adequate, a commodity chip can last a long time.

Re: A 10 year old Xeon is all you need

#125

Post by ryandvm »

PLenz wrote: post #104
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

This. OpenAI and Anthropic are ultimately compute infrastructure plays and not really AI. Everyone will have models, they'll have the ability to run them. This is why the GPU shortage is in their favor.

And like Google and Meta, these companies are going to morph into advertising giants. Advertising is an economic black hole and it eats everything that comes close.

Re: A 10 year old Xeon is all you need

#126

Post by phaser »

skdb476 wrote: post #41
phaser wrote: post #31

What intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop. Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll…

Its a convenience thing. You can run a whole lot of stuff locally from wikipedia to social media/email/video servers whatever. Most people with a full time job and 2 kids dont do it cause who has time and energy to patch and maintain the ever growing complexity of this stuff. These systems will keep growing complex. That also means more bugs. Age old tradeoff between freedom and convenience.

You can run mediawiki at home but you won't have wikipedia. You can run a video server but you won't have all the movies that Netfix has. A local model is actually the real thing.

Re: A 10 year old Xeon is all you need

#127

Post by cduzz »

herval wrote: post #89
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business

This is actually what happens.

I run my word processing software on my apple 2 (a total joke of a computer) instead of running it on the WANG.

I run my book keeping software on visicalc instead of the IBM.

I run my simulation software on my IBM PC (I even paid for the 8087!) instead of the VAX.

Moore's law has, at least so far, allowed the pioneers with toy computers to grow their toys big enough to solve "big boy" problems after some time has allowed the toy computers to be faster and the pioneers have scaled their crappy home-grown solution to solve their 60% of the problem that was originally solved by some enormous complex system.

Eventually the toy infrastructure gets expensive and solves 90-120% of the "big iron" problem space, but it also grows to cost as much as the big iron solution, but then a new generation of toy software and toy systems emerges to disrupt the "big iron" systems.

See also http://www.catb.org/jargon/html/W/wheel-of-reincarnation.htm...

Re: A 10 year old Xeon is all you need

#128

Post by observationist »

herval wrote: post #89
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business

It's a huge difference. If you had AI sufficiently good running locally on a phone, you could devise workflows for things like basic digital hygiene, technical assistance, and tedious tasks like inbox management, image sorting, device updates, and so on. Privacy and security gets a big boost past some local competence threshold, and we're nearly there.

Make the local AI competent enough to do good image generation and editing, realtime voice and music generation, handle agentic tasks with a framework like Hermes, and you can take your AI places to do tasks in contexts that are inaccessible to or inappropriate for cloud.

Frontier big platform models will be the best, but there's a level of "good enough" for local uses that we're already seeing flourish, and "good enough" for the average joe is almost here.

Re: A 10 year old Xeon is all you need

#129

Post by dangus »

RetroTechie wrote: post #106
arpinum wrote: post #25

Earlier quoted context omitted.

How many watts is that setup? Cool you got it to work, but maybe only useful for vintage / retro computing rather than practical if the energy consumption makes it economically wasteful.

How many kWh to fabricate a brand new machine better suited to the task? As long as performance is useable (apply your own metrics!), pulling it from existing hardware is likely the option with the lower eco footprint. Also: chances are it'll only be used for this purpose occasionally, and/or for a short while. In that scenario [fabricating new hardware] always has the bigger eco footprint.

I don’t know why you’d assume that an older system is lower footprint.

If you’ve got something consuming 100 watts average over your 24 hour period, and your electricity costs 20 cents per kWh, you’re already spending almost as much as a Claude subscription.

Just on electricity, this assumes your hardware never fails and you never incur any additional costs.

There’s a big reason why newer more efficient hardware is in demand. Something that’s 10+ years old has drastically worse performance per watt.

Obviously I am not saying to throw away your old hardware as a rule but there is a point where some of this old stuff just isn’t even worth running.

Re: A 10 year old Xeon is all you need

#130

Post by ac29 »

lreeves wrote: post #85

Doesn't accepting 100% of the MTP draft tokens mean you should just be using the smaller model? Usually the acceptance rate in Qwen36 at least is around 60-70% and the "wrong" tokens are still filled in entirely by the base model, but when you just accept 100% of the draft tokens it seems kind of self defeating unless I'm wrong. Also I feel like everyone leaves off prompt processing/prefill speeds in these articles.…

None of those settings set the speculative decoder to accept 100% of drafted token. I assume you are looking at --draft-p-min 0.0, if so, you are misunderstanding what it does.
Post reply on HN