Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

241–250 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#241

Post by jagged-chisel »

yard2010 wrote: post #104
tgma wrote: post #88

Earlier quoted context omitted.

Well I already made the Ctrl+C decision. Yours may have been different, but I suppose only one of us installed it, and that one counts.

I went with the ctrl z approach.

Hopefully you also set it running in the background.

Re: Darkbloom – Private inference on idle Macs

#243

Post by elbac »

tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

I received the same error, but it was followed by this line in the logs, which might explain the lack of inference requests assume there is actual demand...

WARN STT backend failed health check — model will NOT be advertised

Re: Darkbloom – Private inference on idle Macs

#244

Post by p1necone »

kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

Because the "ship software to people, rent their hardware" model has zero up front investment required, presumably. And they don't have to deal with power, cooling, real estate.

Re: Darkbloom – Private inference on idle Macs

#245

Post by dmitrygr »

ta988 wrote: post #240
dmitrygr wrote: post #235

Earlier quoted context omitted.

> Existing machine is no cost for you to run this. That is not at all how modern chips work. Idle chips are mostly powered down, non-idle ones are working and that causes real measurable wear and tear on the silicon. CPU, RAM, NAND all wear and tear measurably with use on current manufacturing processes. https://en.wikipedia.org/wiki/Electromigration

The question is, do they wear faster than they become obsolete, as in much more expensive to run than buying a new one with higher compute/watt. (and you can also factor in the ability to run latest models at usable speed)

It’s complicated. When you design with modern PDKs, you consider the expected duty cycle of the device, expected temps, and the wear and tear on the silicon all together. That affects the layout of the chip as well as certain choices about widths of various features. Generally, one designs consumer SoCs to last 10 years with the expected duty cycle (low). With more wear you could run out of your “years" much faster, maybe even before the warranty.

Re: Darkbloom – Private inference on idle Macs

#246

Post by icedrift »

poorman wrote: post #225

As one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.

Are you actually seeing demand?

Re: Darkbloom – Private inference on idle Macs

#247

Post by LPisGood »

eigengajesh wrote: post #151
kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it. That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted w…

My question is why did you have to design this to use an MDM instead of a simple program running in the terminal or something?
Post reply on HN