Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

61–70 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#61

Post by nnx »

pants2 wrote: post #17

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around…

> My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B.

This seems high. At which quantization? Using LM Studio or something else?

Note: Darkbloom seems to run everything on Q8 MLX.

Re: Darkbloom – Private inference on idle Macs

#63

Post by kennywinker »

znnajdla wrote: post #53
kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

The numbers are obviously high, because if this takes off then the price for inference will also drop. But I still think it’s a solid economic model that benefits low income countries the most. In Ukraine, for example, I know people who live on $200/month. A couple Mac Minis could feed a family in many places. As a business owner, I can think of multiple reasons why a decentralized network is better for me as a busin…

How many of those people who could live off $200USD/month can afford or already have a mac mini in the house?

Re: Darkbloom – Private inference on idle Macs

#64

Post by 0xbadcafebee »

I'm not sure how the economics works out. Pricing for AI inference is based on supply/demand/scarcity. If your hardware is scarce, that means low supply; combine with high demand, it's now valuable. But what happens if you enable every spare Mac on the planet to join the game? Now your supply is high, which means now it's less valuable. So if this becomes really popular, you don't make much money. But if it doesn't become somewhat popular, you don't get any requests, and don't make money. The only way they could ensure a good return would be to first make it popular, then artificially lower the number of hosts.

Re: Darkbloom – Private inference on idle Macs

#65

Post by dnnddidiej »

kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

It is too good to be true. When you see it is making more than a claude code subscription for fuck all work per day.

Prolly gonna make $50 a year tops.

Re: Darkbloom – Private inference on idle Macs

#68

Post by tonyedgecombe »

sethherr wrote: post #46
znnajdla wrote: post #44

Earlier quoted context omitted.

Maybe lunch money for you, but there are people in some parts of the world who live on $200/month. Like Ukraine.

But they probably don’t have M5 MacBook pros idling

Or reliable energy or internet.

Re: Darkbloom – Private inference on idle Macs

#69

Post by Hamuko »

utkarsh_apoorva wrote: post #66

Like the concept. This is not a business - should be an open source GitHub repo maybe. They lost me with just one microcopy - “start earning”. Huge red signal.

But why would I donate my Mac Studio's idle time if I couldn't "start earning"?

Re: Darkbloom – Private inference on idle Macs

#70

Post by thatxliner »

tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

and I don't think they ever will unless they're highly competitive (hopefully that price they have stays? at least for users)

I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand.

I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my original idea was that it was for compute in general, when the main, best use-case, is long(er)-running software like AI training (but I guess inference is long running enough).

But there already exist software out there that lets you rent out your GPU so...

Post reply on HN