31–40 of 69 posts
Does your house have redundant power connections to the grid and a failover generator? That said, my plex server for my friends is on an ups and I'm on 1Gb fiber and I have better uptime than AWS.
You're not serving tens of millions of people.
Many reasons prevent this from being practical for any serious purposes. 1. It depends on what part of the world you are in, but many homes have cooling needs for at least part of the year. The needs to remove excess heat would go up if you are adding more heat -- and it is less efficient to do this at the scale of an individual home than it is at DC scale. 2. Power requirements: While many homelabs have UPS systems…
1. Many places in the world don't ever need cooling 2. If servers are distributed then downtime is distributed, you can virtually guarantee that some of the servers over the world will be online so you can get effectively 100% uptime, something that is not possible in a data center 3. To serve tokens you need very little bandwidth, it's just text in and out 4. All of this is down to the HW and the SW itself, not the…
Then you are likely needing to manage humidity.
Earlier quoted context omitted.
> 1. Many places in the world don't ever need cooling And data centers also exist in cold places. But if you put 8kw of extra heat in someone's home that previously didn't need cooling, it might need it now. > 2. If servers are distributed then downtime is distributed, you can virtually guarantee that some of the servers over the world will be online so you can get effectively 100% uptime, You can! But running more s…
> 1. And data centers also exist in cold places. But if you put 8kw of extra heat in someone's home that previously didn't need cooling, it might need it now. That's the entire point of being in a cold place that you don't need active cooling. Just open the window. > 2. But running more servers with worse uptime is less efficient and requires more capital expense than running fewer servers with better uptime. Even if…
> Even if the cooling is free? Not even free, the cost is negative since it saves heating cost.
Again, having cold air outside is not unique to residential homes. Locating somewhere cold is a strategy for cooling data centers as well. But it doesn’t make environmental management free. You still need to control humidity and move heat. You can’t just run a server outside. However, it isn't the only concern for hosting a compute workload.
> Can you explain the connection of LLMs to power density? This point makes no sense.
The system requirements for a single server to run a frontier workload is a system that would overwhelm the power requirements of practically all residential electrical systems.
> You can lock a box physically
If only ISO 27000/SOC/NIST SP 1800/PCI DSS/ etc were all that easy lol
> Why though if redundancy exists, like you said? Would they still struggle to compete if the cooling cost was effectively negative?
Because of the additional capital costs associated with buying more servers, the additional operational costs of inconveniencing your employees, and the additional operational costs of powering/housing servers that are down.
> You can make cooling cost negative, if that's the majority of the cost, then that's great!
It isn't, most power in a data center is spent to power compute. Even if you do harness waste energy (which some data centers do), it is at best 100% efficient. Residential heat-pumps already have effective efficiencies better than this.
> And you can also place your servers in residential areas with the cheapest power.
And in those places, industrial rates are typically even lower.
Scale is always cheaper.
In France, there are at least two companies that are trying (or tried) to commercialize something with a similar idea : domestic radiators that produce heats from embedded computers that are used as cloud infrastucture. - https://www.hestiia.com/en for the end-user market - https://qarnot.com/en that seems to have since pivoted to low-carbon footprint HPC (was mentionned here -- in French -- as doing computer-based h…
https://www.engie-solutions.com/fr/references/chaleur-fatale...
Does your house have redundant power connections to the grid and a failover generator? That said, my plex server for my friends is on an ups and I'm on 1Gb fiber and I have better uptime than AWS.
> I have better uptime than AWS. You're not serving tens of millions of people.
No private or public entity will grant access to valuable proprietary hardware, as unacceptable risks will not only come from building owners, but also from anyone entering premises.
Also, managing remote nodes evenly spreaded across all areas will be costly. Think of armies of techs on the road permanently, with access problem, dogs or pest barriers, and so on.
A way to solve this would be the allocation of a planned space per block everywhere, which would be safely secured - then available and accessible to all utility organizations: electric, isp, water, phone, data, etc. Heat, power, mini data centers, and such could serve all buildings on a block.
Then other problems emerges: having utilities plan and use these together. Would only work if all services belong to the same entity.
A way around, of course, would be for individuals to setup servers they would own, and rent to data brokers, like Holo project once planned for.
Earlier quoted context omitted.
Part of the problem with current data centers is the density. To make the economics work, you need excessive density, which comes with power, noise, and water requirements. Smaller servers distributed more widely don’t come with the same requirements. They can’t handle all use cases, but something like a Tinybox can run consumer LLM tasks just fine, a SAN with a small server can provide backup storage or storage for…
The density of data centers provides efficiency gains -- if you take the same workload from a high density nvidia DGX setup in a data center and instead distribute it to Tinyboxes running residentially, you'd have an overall net gain in energy use.