[flagged]
A 10 year old Xeon is all you need
221–230 of 301 posts
Re: A 10 year old Xeon is all you need
#222We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.
Happened to me. CoPilot changing prices prompted me to cancel my CoPilot subscription and install a local coding model running entirely in VRAM. Will call Claude APIs when I get really stuck, but I should be able to handle 80% of my needs with a dumber local model. For a long time, too. Programming languages rarely change much, techniques rarely change, so I should be able to use said model for I hope at least five y…
[deleted]
Re: A 10 year old Xeon is all you need
#223Earlier quoted context omitted.
this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business
This is actually what happens. I run my word processing software on my apple 2 (a total joke of a computer) instead of running it on the WANG. I run my book keeping software on visicalc instead of the IBM. I run my simulation software on my IBM PC (I even paid for the 8087!) instead of the VAX. Moore's law has, at least so far, allowed the pioneers with toy computers to grow their toys big enough to solve "big boy" p…
You're right Moore's law has been holding up, but will hit a hard limit on process node size, so all scaling will be based on multiple cores. OTH, computing per watt spent has been plateauing. If the future bottlenecks are energy and cooling, that will require infrastructure-scale solutions. My bet is this is going to be real AI company moat.
Re: A 10 year old Xeon is all you need
#224Earlier quoted context omitted.
It's a huge difference. If you had AI sufficiently good running locally on a phone, you could devise workflows for things like basic digital hygiene, technical assistance, and tedious tasks like inbox management, image sorting, device updates, and so on. Privacy and security gets a big boost past some local competence threshold, and we're nearly there. Make the local AI competent enough to do good image generation an…
Phones and laptops are terrible devices for local AI, way too constrained by bad thermals and small batteries. MiniPC's (many of them using mobile hardware) don't have that particular issue, and can easily run on a 24/7 basis.
[deleted]
Re: A 10 year old Xeon is all you need
#225Glad to see other people realizing this. I've been running Gemma 26B-A4B Q4 on a 2012 Xeon with 16GB to 24GB of RAM in a container. It's getting around 8 to 12 tokens per second. Obviously it's not comparable to huge contexts and running it on a GPU and the image decoder in llama.cpp is super slow compared to a GPU but for some small automation tasks and general trivia questions it's decent. The speed is just enough…
I'm setting up a Frankenstein system at the moment. It's a Chinese DDR3 X99 motherboard with a 12 core Xeon v3, 32gb 1866MT/s ram, and a 1080 Ti.
I'm shoehorning it back in the Optiplex that donated the ram, so it's not ready to go at the moment, but when I had it running on top of the motherboard box as a test I ran the (9B?) gemma4:e4b-it-q4_K_M since it can fit entirely in the 11gb vram. It flew, more than 50tk/s. A model that small isn't useful for coding, but there could be uses. I'd love to figure out a Wake-on-Use and use it as my personal ChatGPT. I'm not sure how that would work... Maybe proxy the LLM thru a Pi with a script to Wake-on-LAN the PC? It'll be a fun weekend project someday.
My always-on LLM is the dense Gemma4:31b that's not quite half in GPU on a 12gb 2060. It's really slow, but the quality is great and my use case is an automated queue so I'm not sitting there watching the output. I have another 2060 but unfortunately the PC won't POST with both installed for some reason.
Re: A 10 year old Xeon is all you need
#226Earlier quoted context omitted.
And like Google and Meta, these companies are going to morph into advertising giants. Advertising is an economic black hole and it eats everything that comes close.
Embedding ads in LLM responses is something researchers are having a lot of trouble figuring out right now. I have seen the results of some early attempts. It fails in such hilarious ways that all these companies are scared of productizing it. But once someone does it, the taboo is broken and everyone else will follow suit immediately.
It's already being done: https://openai.com/index/testing-ads-in-chatgpt/
Re: A 10 year old Xeon is all you need
#227Earlier quoted context omitted.
As five eyes citizen you have at least some rights on paper and you can appeal to your government, but if you are foreigner these guys can go gloves off without any fear of retribution. Try analyzing Epstein files and posting about it, they'll give you a proper penetration test of all your devices to see what you found out about their ex employee. Nowadays even EU citizens migrating away from US cloud providers are a…
Isn't the whole five eyes argument moot because member states spy on citizens from the other countries and trade intel with each other?
No need for that charade if you are a foreigner, even from NATO ally.
Re: A 10 year old Xeon is all you need
#228Earlier quoted context omitted.
this is sorta like saying that being able to run your blog on your laptop will completely implode the cloud business
The primary feature of a blog or any website is that it is available around the clock, that is the primary feature of cloud: around on the clock computer and network that scales on demand. The primary feature of "AI" is to process information and reason with a natural language interface at speed, the primary feature of AI bigboys is to provide the machinery that runs the "models". See the difference?
You severely underestimate how little the fraction of the performance and human labor of a frontier AI is in "the model".
Hosting a blog 24x7 on a laptop is trivial, except for hyperscaling to the front page of HN and Reddit.
Re: A 10 year old Xeon is all you need
#229Earlier quoted context omitted.
> The E5-2620 v4 is great. Have been using it for 10 years now. 10 years? Damn, that is a long time. I always assumed that heat-induced damage will kill a CPU after a certain amount of time (5-7 years). Am I wrong here? I assume yes. Or are CPUs must stronger/tougher than the bad old days?
A quick search on Xeon production yields that it goes through a rather rigorous testing. I wouldn't be surprised that server cpu's in a desktop pc works longer. I can't overclock it either, and that probably helps with its lifespan as well. But yeah, the fact that it actually powers on when i click the button and isn't a limiting factor after 10 years is quite something.
>I can't overclock it either
Except you can overclock v3 :)
Re: A 10 year old Xeon is all you need
#230Earlier quoted context omitted.
For agentic services, how would you be able to tell that you’ve been product-placed?
Hidden advertising is illegal in most jurisdictions, so it has to be indicated to the user for each specific occurrence and hence be trackable anyway.
"AI can make mistakes. Responses include sponsored content or weights."
Now it's compliant with the law.