Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

221–230 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#221

Post by alsetmusic »

washadjeffmad wrote: post #171
stavros wrote: post #157

Earlier quoted context omitted.

You're not taking into account the thermal strain on the machine, though. A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.

Not appreciably, and not before a 5-yr AppleCare+ warranty expires. Out of our >3000 currently active Apple Silicon Macs, failures due to non-physical damage are in the single digits per year. Of those, none have been from production systems with 24/7 uptime and continuous high load, which reflects your parenthetical. Perhaps we haven't met the other end of the bathtub curve yet, but we also won't be retaining any of…

> Not appreciably, and not before a 5-yr AppleCare+ warranty expires.

It’s three years for Macs, though I believe you can pay annually for longer. Five has never been a thing to my knowledge.

Re: Darkbloom – Private inference on idle Macs

#222

Post by bbatsell »

washadjeffmad wrote: post #171
stavros wrote: post #157

Earlier quoted context omitted.

You're not taking into account the thermal strain on the machine, though. A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.

Not appreciably, and not before a 5-yr AppleCare+ warranty expires. Out of our >3000 currently active Apple Silicon Macs, failures due to non-physical damage are in the single digits per year. Of those, none have been from production systems with 24/7 uptime and continuous high load, which reflects your parenthetical. Perhaps we haven't met the other end of the bathtub curve yet, but we also won't be retaining any of…

AppleCare+ annual is perpetual as long as you keep paying it (and Apple offers to switch to that when your 3-year lump sum expires if you choose that instead). I’m guessing it ends whenever they officially discontinue hardware support, which has traditionally been about 7 years after the last unit is produced, but I haven’t reached that yet to know for sure.

Re: Darkbloom – Private inference on idle Macs

#223

Post by pgporada »

liuliu wrote: post #218
dgacmu wrote: post #183

Earlier quoted context omitted.

No provider maintains 100% utilization of GPUs at full rate. Demand is bursty - even if this project is successful, you might expect, e.g., things to be busy during the stock market times when Claude is throwing API errors and then severely underutilized during the same times that Anthropic was offering two-for-one off peak use. And then there's a hit for overprovisioning in general. If the network is not overprovisi…

Hold my beer: https://imgur.com/a/sNAoghL

What's your Y axis?

Re: Darkbloom – Private inference on idle Macs

#224

Post by MyUltiDev »

The hardware-attested privacy path is the interesting part of this, but the economic side has a quieter risk the thread has not named: the load tax per request. MiniMax M2.5 239B from your catalog still has to load all 239B weights even though only 11B are active — that is roughly 120GB at Q4_K_M, and cold load from SSD on Apple Silicon is measurable in tens of seconds. Even the Qwen3.5 122B MoE lands around 65GB cold. If the coordinator routes request number two to a different idle Mac than request number one, or if the owner's machine spun the model out to free memory in between, each request pays that cold load before the first token. Keeping the model resident 24/7 solves the latency but eats into the power budget the operator is trying to amortize in the first place. How does the coordinator decide which provider to keep warm for which model? A 16GB or 32GB home Mac cannot host Qwen3.5 122B MoE at all, and the Mac Studios that can are a much smaller slice of the 100M machine estimate.

Re: Darkbloom – Private inference on idle Macs

#225

Post by poorman »

As one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.

Re: Darkbloom – Private inference on idle Macs

#226

Post by qurren »

poorman wrote: post #225

As one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.

> You can get a 400 watt solar panel on Amazon for $300.

Too expensive. It's probably producing 200 watts average for 8 hours a day. That's 1600 watt hours, which is about $1.60 at PG&E prices. That would take 187 days to recoup the cost of just the panel.

If you include installation costs and "what PG&E steals if you wire it to the same grid" it's probably more like 4x that, which is too long.

Tell me when we can have 400 watt solar panels for $50. Stupid capitalism literally forces solar panel prices to make it unprofitable.

People should never have to take out loans for solar. Solar should be subsidized and forced by the government to be so cheap that it repays for its cost within a month. Then we're talking. Most things I buy to save money, I expect them to repay within a month. Maybe 2 months max.

Re: Darkbloom – Private inference on idle Macs

#227

Post by poorman »

qurren wrote: post #226
poorman wrote: post #225

As one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.

> You can get a 400 watt solar panel on Amazon for $300. Too expensive. It's probably producing 200 watts average for 8 hours a day. That's 1600 watt hours, which is about $1.60 at PG&E prices. That would take 187 days to recoup the cost of just the panel. If you include installation costs and "what PG&E steals if you wire it to the same grid" it's probably more like 4x that, which is too long. Tell me when we can ha…

If you are worried about a $300 solar panel you are not going to like the cost of a Mac Studio M3 Ultra 512 GB! haha

Re: Darkbloom – Private inference on idle Macs

#228

Post by canarias_mate »

dgacmu wrote: post #180

@eigengajesh - Your cost estimator lists Mac Mini M4 Pro with only 24 or 48GB options, but the M4 Pro mini can also be configured with 64GB. At least, I hope so, as I'm typing this on one. ;-) Oh, also, you seem to have some bugs: Gemma: WARN [vllm_mlx] RuntimeError: Failed to load the default metallib. This library is using language version 4.0 which is not supported on this OS. library not found library not found l…

[flagged]

Re: Darkbloom – Private inference on idle Macs

#229

Post by qurren »

poorman wrote: post #227
qurren wrote: post #226

Earlier quoted context omitted.

> You can get a 400 watt solar panel on Amazon for $300. Too expensive. It's probably producing 200 watts average for 8 hours a day. That's 1600 watt hours, which is about $1.60 at PG&E prices. That would take 187 days to recoup the cost of just the panel. If you include installation costs and "what PG&E steals if you wire it to the same grid" it's probably more like 4x that, which is too long. Tell me when we can ha…

If you are worried about a $300 solar panel you are not going to like the cost of a Mac Studio M3 Ultra 512 GB! haha

I'm not "worried" about that cost, I would rather just pay PG&E electricity if the solar panel cost $300.

Just pointing out why capitalism + solar is a failure. Capitalism reprices the good thing to be equally expensive to the bad thing, so that nobody buys the good thing anymore.

Re: Darkbloom – Private inference on idle Macs

#230

Post by sleepybrett »

BuildTheRobots wrote: post #217
eigengajesh wrote: post #151

Earlier quoted context omitted.

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it. That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted w…

I don't worry about bandwidth or constant CPU use, but the one thing that will kill my mac is burning out the SSD. The calculator gives numbers for nearly everything, but I can't obviously see how much space it needs for model storage or how many writes of temp files I should expect if I'm running flat out.

put that stuff on an external disk perhaps, it will eventually crater, but it's easier to replace than macbook internal storage (how are they doing mac minis these days?)
Post reply on HN