Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

181–190 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#181

Post by ori_b »

PunchyHamster wrote: post #169
everforward wrote: post #156

Earlier quoted context omitted.

I work in SRE and the way you describe it would give me pause. The first is that SRE team size primarily scales with the number of applications and level of support. It does scale with hardware but sublinearly, where number of applications usually scales super linearly. It takes a ton less effort to manage 100 instances of a single app than 1 instance of 100 separate apps (presuming SRE has any support responsibiliti…

> The last is that I don’t know many SREs (maybe none at all) that are well-versed enough in all the hardware disciplines to manage a footprint the size we’re talking. If each SRE is 4 racks and a minimum team size is 4, that’s 16 racks. You’d need each SRE to be comfortable enough with networking, storage, operating system, compute scheduling (k8s, VMWare, etc) to manage each of those aspects for a 16 rack system. I…

Noteworthy: the number of apps isn't affected by whether the machines are in your datacenter or Amazon's.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#182

Post by to11mtm »

MengerSponge wrote: post #11
rubyn00bie wrote: post #6

I’ve not kept up with Intel in a while, but one thing that stood out to me is these are all E cores— meaning no hyperthreading. Is something like this competitive, or preferred, in certain applications? Also does anyone know if there have been any benchmarks against AMDs 192 core Epyc CPU?

I don't know the nitty-gritty of why, but some compute intensive tasks don't benefit from hyperthreading. If the processor is destined for those tasks, you may as well use that silicon for something actually useful. https://www.comsol.com/support/knowledgebase/1096

It's a few things; mostly along the lines of data caching (i.e. hyper threading may mean that other thread needs a cache sync/barrier/etc).

That said I'll point to the Intel Atom - the first version and refresh were an 'in-order' where hyper-threading was the cheapest option (both silicon and power-wise) to provide performance, however with Silvermont they switched to OOO execution but ditched hyper threading.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#183

Post by evanjrowley »

MostlyStable wrote: post #78
mort96 wrote: post #5

Earlier quoted context omitted.

I bought a bundle with 512GB of RAM and an older 24-core EPYC (7F72) + supermicro motherboard on ebay a bit over a year ago, it was really an amazing deal and has made for a truly nice NAS. If you're okay with stuff that's old enough that you can buy decommissioned server stuff, you can get really high-quality gear at surprisingly low prices. Companies decommission hardware on a schedule after all, not when it stops…

I've heard it claimed that the era of being able to do this (buy slightly old used server hardware cheap on ebay) is coming to an end because, in the quest for ever more efficiency, the latest server hardware is no longer compatible with off-the-shelf power supplies etc. (there was more but that's the part that I remember) and therefore won't have any value on the second hand market. I hope it was wrong, but it seems…

The power supply incompatibility came to fruition a long time ago. Buying used Supermicro ATX motherboards to build into servers stopped being a thing for me about 1 decade ago. But used servers and desktops, even with their non-standard parts, have continued to deliver high value for me even today.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#184

Post by charcircuit »

ocdtrekkie wrote: post #144
CyberDildonics wrote: post #46

Earlier quoted context omitted.

Is infra really that hard to set up? It seems like infra is something a infra expert could establish to get the infra going and then your infra would be set up and you would always have infra.

As a big on-prem guy, I think cloud makes sense for early startups. Lead time on servers and networking setup can be significant, and if you don't know how much you need yet you will either be resource starved or burn all your cash on unneeded capacity. On-prem wins for a stable organization every time though.

You can rent a vps or dedicated server if you need something immediately to without going to cloud providers.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#185

Post by fc417fc802 »

epistasis wrote: post #14
mort96 wrote: post #5

Earlier quoted context omitted.

I bought a bundle with 512GB of RAM and an older 24-core EPYC (7F72) + supermicro motherboard on ebay a bit over a year ago, it was really an amazing deal and has made for a truly nice NAS. If you're okay with stuff that's old enough that you can buy decommissioned server stuff, you can get really high-quality gear at surprisingly low prices. Companies decommission hardware on a schedule after all, not when it stops…

RAM! (And NAND SSDs too now, probably...) When I was looking in October, I hadn't bought hardware for the better part of a decade, and I saw all these older posts on forums for DDR4 at $1/GB, but the lowest I could find was at least $2/GB used. These days? HAH! If I had a decent sales channel I might be speculating on DDR4/DDR5 RAM and holding it because I expect prices to climb even higher in the coming months.

I don't remember DDR4 ever hitting $1/GB. Was I not shopping in the right places? IIRC DDR3 settled in at $1/GB quite a long time ago and recycled datacenter DDR4 was maybe ballpark $2.50/GB at some point.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#186

Post by PunchyHamster »

renewiltord wrote: post #16

Core density plus power makes so many things worthwhile. Generally human cost of managing hardware scales with number of components under management. CPUs very reliable. So once you get lots of CPU and RAM on single machine you can run with very few. But right pricing hardware is hard if you’re small shop. My mind is hard-locked onto Epyc processors without thought. 9755 on eBay is cheap as balls. Infinity cores! Pro…

> Previously skillset was required. These days you plug Ethernet port, turn on Claude Code dangerously skip permissions “write a bash script that is idempotent that configures my Mikrotik CCR, it’s on IP $x on interface $y”. Hotspot on. Cold air blowing on face from overhead coolers. 5 minutes later run script without looking. Everything comes up.

Last time I tried to do anything networking with Claude it set up route preference in opposite order (it thought lower number means more preferred, while it was opposite), fucking it up completely, and then invented config commands that do not exist in BIRD (routing software suite).

Then I looked at 2 different AIs and they both hallucinated same BIRD config commands that were nonexistent. And by same I mean they hallucinated existence of same feature.

> If your teams have to go through IT to provision machine and IT have to go through finance so that spend is reliable, everybody slows down too much. You can’t just spin up next product.

The time of having to order a bunch of servers for new project is long over. We just spun k8s cluster for devs to self-service themselves and the prod clusters just have a bit of accounting shim so adding new namespace have to be assigned to a certain project so we can bill client for it.

Also you're allowed to use cloud services while you have on-prem infrastructure. You get best of both, with some cognition cost involved.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#187

Post by fc417fc802 »

MostlyStable wrote: post #78
mort96 wrote: post #5

Earlier quoted context omitted.

I bought a bundle with 512GB of RAM and an older 24-core EPYC (7F72) + supermicro motherboard on ebay a bit over a year ago, it was really an amazing deal and has made for a truly nice NAS. If you're okay with stuff that's old enough that you can buy decommissioned server stuff, you can get really high-quality gear at surprisingly low prices. Companies decommission hardware on a schedule after all, not when it stops…

I've heard it claimed that the era of being able to do this (buy slightly old used server hardware cheap on ebay) is coming to an end because, in the quest for ever more efficiency, the latest server hardware is no longer compatible with off-the-shelf power supplies etc. (there was more but that's the part that I remember) and therefore won't have any value on the second hand market. I hope it was wrong, but it seems…

When boxes get decommissioned it's generally the entire thing. So you can pick up used power supplies as well. Or just buy new because even if it isn't ATX it's still a widely produced item that's used across multiple product lines.

The troublesome hardware is the stuff with custom backplanes and multiple daughterboards each hosting a node. Also AMD CPUs that lock themselves to a single motherboard.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#188

Post by wmf »

fc417fc802 wrote: post #176
jeffbee wrote: post #60

Earlier quoted context omitted.

There definitely are bottlenecks. The one I always think of is the kernel's networking stack. There's no sense in using the kernel TCP stack when you have hundreds of independent workloads. That doesn't make any more sense than it would have made 20 years ago to have an external TCP appliance at the top of your rack. Userspace protocol stacks win.

Do the partitioned stacks of network namespaces share a single underlying global stack or are they fully independent instances? (And if not, could they be made so?)

Usually network namespaces are linked together with a single bridge so you can get lock contention there.

If you have a separate physical NIC for each namespace you probably won't have any contention.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#189

Post by jeffbee »

wmf wrote: post #188
fc417fc802 wrote: post #176

Earlier quoted context omitted.

Do the partitioned stacks of network namespaces share a single underlying global stack or are they fully independent instances? (And if not, could they be made so?)

Usually network namespaces are linked together with a single bridge so you can get lock contention there. If you have a separate physical NIC for each namespace you probably won't have any contention.

I think you could get much of the way there by isolating a single NIC's receive queues, so the kernel doesn't decide to run off and service softirqs for random foreign tasks just because your task called tcp_sendmsg.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#190

Post by stego-tech »

Aurornis wrote: post #114
tgrowazay wrote: post #107

Earlier quoted context omitted.

Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, th…

$120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that…

Out of all the comments on numbers, SREs, and scaling, you get the response for meeting numbers with numbers!

> $120K isn't going to cover the fully loaded costs of an SRE who can set up and run that.

Literally this. I can do SRE on-prem and cloud, and my 50/30/20 budget break-even point (as in, needs and savings but no wants - so 70%) is $170k before taxes. Rent is astonishingly high right now, and the sort of mid-career professional you want to handle SRE for your single DC is going to take $150k in this market before fucking off to the first $200k job they get.

Know your market, and pay accordingly. You cannot fuck around with SREs.

> Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever.

This is less of an issue than you might think, but strongly dependent upon the quality of talent you’ve retained and the budget you’ve given them. Shitbox hardware or cheap-ass talent means you’ll need to double or triple up locally, but a quality candidate with discretion can easily be supported by a counterpart at another office or site, at least short-term. Ideally though, yeah, you’ll need two engineers to manage this stack, but AWS savings on even a modest (~700 VMs) estate will cover their TC inside of six months, generally.

> There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one. You can start diagnosing things and replacing parts and hope it gets to the root issue, but that's more downtime.

This strikes at another workload I neglected to mention, and one I highly recommend keeping in the public cloud: GPUs.

GPUs on-prem suck. Drivers are finnicky, firmware is flakey, vendor support inconsistent, and SR-IOV is a pain in the ass to manage at scale. They suck harder than HBAs, which I didn’t think was possible.

If you’re consuming GPUs 24x7 and can afford to support them on-prem, you’re definitely not here on HN killing time. For everyone else, tune your scaling controls on your cloud provider of choice to use what you need, when you need it, and accept the reality that hyperscalers are better suited for GPU workloads - for now.

> Going on-prem like this is highly risky.

Every transaction is risky, but the risk calculus for “static” (ADDS) or “stable” (ERP, HRIS, dev/test) work makes on-prem uniquely appealing when done right. Segment out your resources (resist the urge for HPC or HCI), build sensible redundancies (on-prem or in the cloud), and lean on workhorse products over newer, fancier platforms (bulletproof hypervisors instead of fragile K8s clusters), and you can make the move successful and sensible. The more cowboy you go with GPUs, K8s, or local Terraform, the more delicate your infra becomes on-prem - and thus the riskier it is to keep there.

Keep it simple, silly.

Post reply on HN