Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

71–80 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#71

Post by 59nadir »

chaoz_ wrote: post #9

That solution actually makes great sense. So Apple won in some strange way again? Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. I think batch-evals for non-sensitive data has great PMF here.

> So Apple won in some strange way again?

Heh, what did they win exactly? This is just a way for another company to extract value out of the single region of the world where Apple is a relevant vendor, and it happens to be the one where it's the easiest to pull people into schemes.

Re: Darkbloom – Private inference on idle Macs

#72

Post by tgma »

thatxliner wrote: post #70
tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

and I don't think they ever will unless they're highly competitive (hopefully that price they have stays? at least for users) I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand. I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my orig…

People underestimate how efficient cost/token is for beefy GPUs if you are able to batch. Unlikely for one off consumer unit to be able to compete long term.

Re: Darkbloom – Private inference on idle Macs

#74

Post by starkeeper »

thatxliner wrote: post #70
tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

and I don't think they ever will unless they're highly competitive (hopefully that price they have stays? at least for users) I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand. I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my orig…

What's a good place to do this?

Re: Darkbloom – Private inference on idle Macs

#75

Post by lxglv »

tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

weird to learn that they do not generate inference requests to their network themselves to motivate early adopters at least to host their inference software

Re: Darkbloom – Private inference on idle Macs

#76

Post by agnosticmantis »

kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

"You could see a single robotaxi being worth, or providing, about $30,000 of gross profit per year. ... A Tesla is an appreciating asset..."

- Elon Musk during Tesla's Autonomy Day in April 2019.

Re: Darkbloom – Private inference on idle Macs

#78

Post by splittydev »

tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

They released this like a day ago, I'm not surprised that there's not enough demand right now. Give it some time to take off

Re: Darkbloom – Private inference on idle Macs

#80

Post by tgma »

splittydev wrote: post #78
tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

They released this like a day ago, I'm not surprised that there's not enough demand right now. Give it some time to take off

You'd think to bootstrap a marketplace you'd spend your own money to feed fake requests (or perhaps allow free chat so that they induce requests).

Still, absolute zero is an unacceptable number. Had this running for more than an hour.

Post reply on HN