Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serving AI from the Basement – 192GB of VRAM Setup

ahmadosman.com

Post reply on HN
Print view View original

151–160 of 279 posts

Re: Serving AI from the Basement – 192GB of VRAM Setup

#151

Post by NavinF »

Dylan16807 wrote: post #144
NavinF wrote: post #141

Earlier quoted context omitted.

In the bay area every home I've lived in had 20A circuits for everything. The NEC 80% rule is only relevant in data centers where someone enforces it. The time-current curve for any 20A breaker shows that it will take infinite time to trip when you draw 20A. This is correct behavior because 12awg wire can handle 20A indefinitely

Do they actually install the 20A outlets with the sideways part? I don't know if I've ever seen one in person. I guess the outlet doesn't matter a whole lot, especially if you take multiple cables to the wall. But I still don't want to be doing unenforced electrical violations.

Some outlets have the 20A sideways part, but from a safety/code perspective that's irrelevant.

Consider what happens when you plug two 10A devices into a power strip. In the US there are no true 15A rated outlets. All outlets must handle 20A because the electrical code allows any outlet to be on a 20A breaker.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#152

Post by Dylan16807 »

NavinF wrote: post #151
Dylan16807 wrote: post #144

Earlier quoted context omitted.

Do they actually install the 20A outlets with the sideways part? I don't know if I've ever seen one in person. I guess the outlet doesn't matter a whole lot, especially if you take multiple cables to the wall. But I still don't want to be doing unenforced electrical violations.

Some outlets have the 20A sideways part, but from a safety/code perspective that's irrelevant. Consider what happens when you plug two 10A devices into a power strip. In the US there are no true 15A rated outlets. All outlets must handle 20A because the electrical code allows any outlet to be on a 20A breaker.

> Consider what happens when you plug two 10A devices into a power strip.

I don't know about any argument that depends on such an action being safe. There's plenty of power strips I wouldn't trust with 20 amps.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#153

Post by NavinF »

Dylan16807 wrote: post #152
NavinF wrote: post #151

Earlier quoted context omitted.

Some outlets have the 20A sideways part, but from a safety/code perspective that's irrelevant. Consider what happens when you plug two 10A devices into a power strip. In the US there are no true 15A rated outlets. All outlets must handle 20A because the electrical code allows any outlet to be on a 20A breaker.

> Consider what happens when you plug two 10A devices into a power strip. I don't know about any argument that depends on such an action being safe. There's plenty of power strips I wouldn't trust with 20 amps.

That's just how the US electrical code works. I assure you that every "15A" outlet on the wall can handle 20A. That's why products like this don't have a fuse: https://www.homedepot.com/p/Woods-2-ft-12-3-SJTW-Multi-Outle...

Re: Serving AI from the Basement – 192GB of VRAM Setup

#154

Post by nullindividual »

disiplus wrote: post #123
nullindividual wrote: post #46

Earlier quoted context omitted.

The reason I asked is I used to run a dual X5650 server with SSDs and it was about $50/month with the cheapest (or very close to) rates in the US.

We have more expensive gas then usa, but i pay like a 5 cent per kwh @220V Did not know how expensive it is in usa, especially California.

I pay 10 cents per KWH. Most of the electricity in my state is produced by hydropower.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#155

Post by phil21 »

wpietri wrote: post #121
tcdent wrote: post #70

Earlier quoted context omitted.

I can't believe a group of engineers are so afraid of residential power. It is not expensive, nor is it highly technical. It's not like we're factoring in latency and crosstalk... Read a quick howto, cruise into Home Depot and grab some legos off the shelf. Far easier to figure out than executing "hello world" without domain expertise.

Ah yes, the "move fast and burn your house down" school of "engineering".

Adding a bog-standard breaker and a short conduit run is about as simple as it gets for electric work. It’s rather low risk if you simply read the code and follow it.

If you know nothing about basic electric work or principles, sure - spend the $500 to have an electrician add a 30 or 50A 220V outlet near your electric service panel. Totally reasonable to do as it is indeed dangerous to touch things you don’t understand.

It’s far less complex and less dangerous than adding an EV charge point to your garage which seems to be quite common for this crowd. This is the same (less, since you typically have a lot more flexibility on where to locate the outlet and likely don’t need to pull through walls) complexity as adding a drop for an electric stove.

Where the “home electric hackers” typically tend to get in trouble is doing stuff like adding their own generator connection points and not properly doing interlocks and all that fun stuff.

If you can replace your own light switches and wall receptacles you are just one step away from adding an additional branch circuit. Lots of good learning material out there on the subject these days as well!

Re: Serving AI from the Basement – 192GB of VRAM Setup

#156

Post by rootusrootus »

beAbU wrote: post #137
lolinder wrote: post #116

Earlier quoted context omitted.

We have an electric kettle in the US and it runs just fine drawing 1500W. You're correct that the dryer is on a larger circuit, though.

> and it runs just fine drawing 1500W. You think that this is "just fine" because you've never experienced the glory that is a 3kW kettle!

I just get 99C water from a tap next to my kitchen sink. Why do people still use kettles?

Re: Serving AI from the Basement – 192GB of VRAM Setup

#157

Post by metadat »

manav wrote: post #136
Tepix wrote: post #115

So, how do you connect the 8th card if you have 7 PCIe 4.0 x16 slots available?

PCIe bifurcation - so splitting one of the x16 slots into two x8 or similar.

Worth mentioning - this also cuts the available bandwidth to each card by 50%.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#158

Post by rootusrootus »

disiplus wrote: post #123
nullindividual wrote: post #46

Earlier quoted context omitted.

The reason I asked is I used to run a dual X5650 server with SSDs and it was about $50/month with the cheapest (or very close to) rates in the US.

We have more expensive gas then usa, but i pay like a 5 cent per kwh @220V Did not know how expensive it is in usa, especially California.

US generally has some of the cheapest electricity, about half of what Europeans pay. California has some areas that are abnormally expensive.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#159

Post by buildbot »

ganoushoreilly wrote: post #143
elorant wrote: post #140

The motherboard has 7 PCie slots and there are 8 GPUs. So where does the spare one connect to? Is he using two GPUs in the same slot limiting the bandwidth?

may be using an nvme to pcie adapter, common in the crypto mining world

It’s an epyc server board, it probably has actual U.2/MCIO pcie ports on the board that can be merged back into a 16x slot in the bios. I had/have several boards like that.

Re: Serving AI from the Basement – 192GB of VRAM Setup

#160

Post by [deleted] »

rootusrootus wrote: post #156
beAbU wrote: post #137

Earlier quoted context omitted.

> and it runs just fine drawing 1500W. You think that this is "just fine" because you've never experienced the glory that is a 3kW kettle!

I just get 99C water from a tap next to my kitchen sink. Why do people still use kettles?

[deleted]
Post reply on HN