Building an AI server on a budget • Hacker News Board

Re: Building an AI server on a budget

#111

Post by hadlock » Tue, Jun 10, 2025, 7:33 PM UTC

usercvapp wrote: post #37

I have a server at home sitting IDLE for the last 2 years with 2 TB of RAM and 4 CPUs. I am gonna push it this week and launch some LLM models to see how they perform! How much electric bill efficient are they running locally?

Unless it's actively processing something it's also sitting idle, so pretty efficient besides it vacuuming up all your system memory.

Re: Building an AI server on a budget

#112

Post by throwaway314155 » Tue, Jun 10, 2025, 10:12 PM UTC

memcg wrote: post #99

throwaway314155 wrote: post #24

Earlier quoted context omitted.

Eh, yeah - the article starts off pretty specific but then gets into the weeds of stuff like how to put your PC together, which is far from novel information and certainly not on-topic in my opinion.

I sent the article link to my son because he does not have experience building or assembling hardware or installing or using Linux. Also took the author's ChatGPT prompt and changed it to ask about reusing two HPE ML150 Gen9 servers I picked up free. I think my son will benefit from the details in the article that many find off-topic.

That's totally fair! My personal experience with this was growing up in the aughts and building my first gaming PC from scratch. I associate that as being far more accessible than going all-in on an AI server. Of course in this instance, the two are effectively the same.

Re: Building an AI server on a budget

#113

Post by itake » Thu, Jun 12, 2025, 7:33 AM UTC

mythz wrote: post #82

itake wrote: post #76

Earlier quoted context omitted.

My home server detects NSFW images in user generated content on my side project. source code: https://github.com/KevinColemanInc/NSFW-FLASK

Cool, I've tried a few but settled on using EraX NSFW to do the same.

just wondering, why that one? it looks small, which is probably doesn't require a gpu.

The dataset seems to be images of high production value (e.g. limited races, staged poses, etc). If I have time, I will compare it with Bumble's model, but I think the images I'm trying to identify are closer to Bumble's training set.