Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A 10 year old Xeon is all you need

point.free

Post reply on HN
Print view View original

131–140 of 301 posts

Re: A 10 year old Xeon is all you need

#131

Post by zozbot234 »

observationist wrote: post #128
herval wrote: post #89

Earlier quoted context omitted.

this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business

It's a huge difference. If you had AI sufficiently good running locally on a phone, you could devise workflows for things like basic digital hygiene, technical assistance, and tedious tasks like inbox management, image sorting, device updates, and so on. Privacy and security gets a big boost past some local competence threshold, and we're nearly there. Make the local AI competent enough to do good image generation an…

Phones and laptops are terrible devices for local AI, way too constrained by bad thermals and small batteries. MiniPC's (many of them using mobile hardware) don't have that particular issue, and can easily run on a 24/7 basis.

Re: A 10 year old Xeon is all you need

#132

Post by phaser »

mr_toad wrote: post #81
phaser wrote: post #31

What intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop. Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll…

This has always been true of software, particularly games. You can get a 5-6 year old game for a fraction of the price, and run it on modest hardware. But the industry wont sit on its hands for 5 years, there will be newer software that requires better hardware.

Technology doesn't always work like that.

A new game is a totally new world with everything created from scratch. A creation. A model, on the other hand, is a reinterpretation machine for hundreds of years of human creations, but not a creation in itself, more like a discovery.

You would think that by now we would have a much better Bitcoin that's taking over the payment networks of the world but what we actually got is a shitload of shitcoin.

Re: A 10 year old Xeon is all you need

#133

Post by asdfsa32 »

herval wrote: post #89
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business

The primary feature of a blog or any website is that it is available around the clock, that is the primary feature of cloud: around on the clock computer and network that scales on demand.

The primary feature of "AI" is to process information and reason with a natural language interface at speed, the primary feature of AI bigboys is to provide the machinery that runs the "models".

See the difference?

Re: A 10 year old Xeon is all you need

#134

Post by Scoundreller »

malmz wrote: post #112
herval wrote: post #89

Earlier quoted context omitted.

this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business

Running an LLM locally is theoretically viable. Running your blog on your laptop is never viable (unless you hook it up like a server). One just requires compute while the other a stable network.

tbh, my home network is pretty close to the stability of my host these days…

But my downtimes are a bit self-inflicted: changing ISPs which I can personally workaround but harder for a blog where one expects uptime.

Re: A 10 year old Xeon is all you need

#136

Post by brookst »

PLenz wrote: post #104
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

This. OpenAI and Anthropic are ultimately compute infrastructure plays and not really AI. Everyone will have models, they'll have the ability to run them. This is why the GPU shortage is in their favor.

How does that view align with Anthropic leasing data centers from others?

I don’t know OpenAI’s infra, but to the extent they are buying GPUs and building data centers with their own money, that sounds like a bad move.

Satya has mismanaged the AI transition in many ways, but one thing he got right is that models are commodities, and the value is in applications that apply them to create user benefit. I agree that any company trying to build a moat with a model is not long for this world.

Re: A 10 year old Xeon is all you need

#137

Post by mwpmaybe »

dark-star wrote: post #86
cafkafk wrote: post #2

Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma 4) running at reading speed on an old recycled server with a single Xeon E5-2620 v4 and 128GB of DDR3 RAM (and no GPU). It took a lot of work, but it actually worked out somehow.…

Something doesn't add up here. As someone who has only recently built a home-server from an E5-26xx v2 on DDR3 RAM (because I have a sh*tload of 32g DDR3 DIMMs), I can confidently say that the newer cores (E5-26xx v3 and v4) only run on DDR4 memory... So either you have a v2 instead of a v4 (and run on DDR3 memory), or you have a v4 but with DDR4 memory (not DDR3) Everything else doesn't work

There are some OEM-only v3/v4 parts with dual memory controllers (because of a RAM supply crunch at the time, funnily enough), but the E5-2620 v4 is not one of them. The classic example is the very popular 12-core E5-2678 v3.

Re: A 10 year old Xeon is all you need

#138

Post by brookst »

IdiotSavage wrote: post #117
squidbeak wrote: post #110

Earlier quoted context omitted.

> Otherwise it would be like Intel and Microsoft had decided in the year 2000 that computers are "good enough" now and we would have explored what's possible with that hardware ever since. I think you've misunderstood what good enough means in the context - which is a model capable of completing the tasks assigned to it without having the breadth of full generalization. Your analogy breaks down because of this - we d…

I think you've misunderstood the analogy. Just ignore it, analogies mostly break down anyways. > a model capable of completing the tasks assigned to it The thing is, the "task assigned to it" is changing with improved capabilities. If everyone around you in 2036 is using general AI to do amazing stuff, you will probably have little interest in vibe coding slop like it's 2026.

Analogies are like metaphors, they’re illustrative rather than literal.

Re: A 10 year old Xeon is all you need

#139

Post by Scoundreller »

ryandvm wrote: post #123
cmiles8 wrote: post #84

We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.

I disagree. We are currently in a weird period where these frontier AI companies are losing tons of money even on the subscription-based AI models. It's just too compute intensive and there's no way most people are going to be buying the kind of hardware required to run $20 worth of inference every day. Sadly - it's going to be ads. Advertising is going to get in there and enshittify the whole thing because as always…

Ads are usually the workaround where you don’t deliver enough value to get people to subscribe or payments are unavailable for some reason.

It makes sense to show some ads and get some money at low volume (like a faraway reader wanting to read a story in your local newspaper) but taking money from regular users directly will pay much more.

Newspapers are happy to cannibalize 99% of their ad revenue with a paywall if that 1% subscribes because that’s how much more money you make from someone paying $10-$20/month vs ads.

But yeah, if people use it as a buying recommendation engine, that’s where the money is on ads/referrals but a lot of AI use has little/no connection to buying intent touchpoints.

Re: A 10 year old Xeon is all you need

#140

Post by mwpmaybe »

bobmcnamara wrote: post #83
vhaudiquet wrote: post #16

The E5 2620-v4 only supports DDR4.

Probably in an x99 motherboard

The memory controller is integrated into the CPU, so the motherboard chipset is irrelevant. There are some OEM-only v3/v4 parts with dual memory controllers, but the E5-2620 v4 is not one of them.
Post reply on HN