271–280 of 301 posts
remember if you serve real customers as a bootstrapped business - you can afford the whole serve down for maintenance. no need for 99.999%.
better than hetzner.
Earlier quoted context omitted.
Hidden advertising is illegal in most jurisdictions, so it has to be indicated to the user for each specific occurrence and hence be trackable anyway.
"AI can make mistakes. Responses include sponsored content or weights." Now it's compliant with the law.
I've got an old HP Z-620 workstation with dual E5-2697 v2 CPUs (24 cores total, 48 threads @ 2.7GHz) and 128GB of DDR3 RAM. The docs say it supports up to 192GB, but I wasn't able to get it to POST with all the RAM slots full. It's still a "homelab" beast and does great with development and GIS/Mapping applications. I was not able to figure out how to run AI workloads on it with decent performance, however, so I fina…
Earlier quoted context omitted.
This is not true. A few well known brands made both DDR3 and DDR4 servers that support v3 & v4 chips. Ask me how I know :-)
enlighten us
We’re not there yet, but the obvious endgame of the present bubble insanity is open models running on local hardware and devices are “good enough” for most use cases. That will completely implode what’s going on at the moment in tech.
Not saying this isn't the case, but my Anthropic subscription costs me less than the electricity would to power such a home inference system.
What intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop. Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll…
--what this means for the valuation of the AI companies Probably nothing. Most users have no idea what an LLM is or how it runs. Anecdotally speaking, I see many LLM users default to whatever their day job provides to them. And even slightly more sophisticated users seem ok with paying for their openai or anthropic subscriptions. Maybe we will see a small but dedicated group of open weight model users who prefer loca…
I think there's actually a big market opportunity here. Somebody, like Dell or HP, should start selling turnkey on-prem LLM servers.
Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma 4) running at reading speed on an old recycled server with a single Xeon E5-2620 v4 and 128GB of DDR3 RAM (and no GPU). It took a lot of work, but it actually worked out somehow.…
I bought a renewed 2x E5-2690v4 server (28c/56t) 128gb on amazon for under $500 2 years ago (28c/56t) dell T7810 search amazon for "chia farming" ...and scroll past chia seeds :) now same machine is 2.5x the price https://www.amazon.com/dp/B095TRGCSX but way cheaper than current ddr5 machines
2.5x?! I have a bunch of older Haswell servers I got for free that are rotting away in my garage. I had initially thought of stripping out the ECC DDR4, but now I'm wondering if I'll get takers on Marketplace...
Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma 4) running at reading speed on an old recycled server with a single Xeon E5-2620 v4 and 128GB of DDR3 RAM (and no GPU). It took a lot of work, but it actually worked out somehow.…
I bought a renewed 2x E5-2690v4 server (28c/56t) 128gb on amazon for under $500 2 years ago (28c/56t) dell T7810 search amazon for "chia farming" ...and scroll past chia seeds :) now same machine is 2.5x the price https://www.amazon.com/dp/B095TRGCSX but way cheaper than current ddr5 machines
I have a 3060 12gb card I'd love to hook up to my PoE Reolink cameras for face detection and to get off of the Reolink app.
How about the iMac Pro? Would that work? I was able to put 128gb in it (not as easy as the regular iMac but possible).
I've been running various models on a Mac Pro 2013 (8 cores, 32 GB RAM) at about 8 to 10 t/s for months. It's not fast, but it's more than enough for many actual tasks, in particular background tasks. An iMac pro will do just as well I suppose.