Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

171–180 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#171

Post by PunchyHamster »

Aurornis wrote: post #114
tgrowazay wrote: post #107

Earlier quoted context omitted.

Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, th…

$120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that…

> There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one. You can start diagnosing things and replacing parts and hope it gets to the root issue, but that's more downtime.

they come with warranty, often with technican guaranteed to arrive within few hours or at most a day. Also if SHTF just getting cloud to augument current lackings isn't hard

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#172

Post by to11mtm »

Retr0id wrote: post #141
zokier wrote: post #84

Earlier quoted context omitted.

afaik the mainline limit is 4096 threads. HP sells server with 32 sockets x 60 cores/socket x 2 threads/core = 3840 threads, so we are pretty close to that limit.

I had no idea we had socket counts so high, do you know where I could find a picture of one?

Sounds like a HPE Compute Scale-up Server 3200, but again keep in mind that's something where there's probably a fabric between nodes one way or another.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#173

Post by PunchyHamster »

LunaSea wrote: post #91
ozgrakkurt wrote: post #85

Earlier quoted context omitted.

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of…

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

> With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

we have 7 racks and 3 people. The things you mentioned aren't even 5% of the workload.

There are things you figure out once, bake into automation, and just use.

You install server once and remove it after 5-10 years, depending on how you want to depreciate it. Drives die rarely enough it's like once every 2 months event at our size

The biggest expense is setting up automation (if I was re-doing our core infrastructure from scratch I'd probably need good 2 months of grind) but after that it's free sailing. Biggest disadvantage is "we need a bunch of compute, now", but depending on business that might never be a problem, and you have enough savings to overbuild a little and still be ahead. Or just get the temporary compute off cloud.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#174

Post by to11mtm »

skyberrys wrote: post #145
avhception wrote: post #139

A bad moment to have a make-or-break moment for your CPU business - a lot of customers will probably hold off purchases right now because of the RAM prices, no matter how good your CPU might be.

Isn't this new server CPU a drop in replacement though? So the DC could pull off the old CPU, drop in the new one and not touch the existing RAM setup, yet be able to deliver better performance within the limits of the existing RAM. Then once RAM prices drop (okay that might be a while) separately upgrade the RAM at a different time.

That's semi-dependent on supplier arrangements; i.e. lots of shops won't want to upgrade CPUs on a server out of fear that they can't get support later; sometimes that's justified by contract, sometimes it's not.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#175

Post by PunchyHamster »

citrin_ru wrote: post #128
carefree-bob wrote: post #58

Earlier quoted context omitted.

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

> price of acquiring talent to manage the gear Is it still a problem in 2026 when unemployment in IT is rising? Reasons can be argued (the end of ZIRP or AI) but hiring should be easier than it was at any time during the last 10 years.

people with that set of skills are never looking for job for long.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#176

Post by fc417fc802 »

jeffbee wrote: post #60
50lo wrote: post #27

With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU. I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

There definitely are bottlenecks. The one I always think of is the kernel's networking stack. There's no sense in using the kernel TCP stack when you have hundreds of independent workloads. That doesn't make any more sense than it would have made 20 years ago to have an external TCP appliance at the top of your rack. Userspace protocol stacks win.

Do the partitioned stacks of network namespaces share a single underlying global stack or are they fully independent instances? (And if not, could they be made so?)

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#177

Post by esseph »

PunchyHamster wrote: post #169
everforward wrote: post #156

Earlier quoted context omitted.

I work in SRE and the way you describe it would give me pause. The first is that SRE team size primarily scales with the number of applications and level of support. It does scale with hardware but sublinearly, where number of applications usually scales super linearly. It takes a ton less effort to manage 100 instances of a single app than 1 instance of 100 separate apps (presuming SRE has any support responsibiliti…

> The last is that I don’t know many SREs (maybe none at all) that are well-versed enough in all the hardware disciplines to manage a footprint the size we’re talking. If each SRE is 4 racks and a minimum team size is 4, that’s 16 racks. You’d need each SRE to be comfortable enough with networking, storage, operating system, compute scheduling (k8s, VMWare, etc) to manage each of those aspects for a 16 rack system. I…

So your definition of SRE is anybody that works on infra?

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#178

Post by PunchyHamster »

CyberDildonics wrote: post #46
jmward01 wrote: post #37

Earlier quoted context omitted.

Cloud = the right choice when just starting. It isn't about infra cost, it is about mental cost. Setting up infra is just another thing that hurts velocity. By the time you are serving a real load for the first time though you need to have the discussion about a longer term strategy and these points are valid as part of that discussion.

Is infra really that hard to set up? It seems like infra is something a infra expert could establish to get the infra going and then your infra would be set up and you would always have infra.

You are correct but it still takes time. You can start using cloud today but you need to:

* sign the papers for server colo * get quote and order servers (which might take few weeks to deliver!), near always a pair of switches * set them up, install OSes, set up basic services inside the network (DNS, often netboot/DHCP if you want to have install over network, and often few others like image repository, monitoring etc.)

It's "we have product and cashflow, let's give someone a task to do it" thing, not "we're a startup ,barely have PoC" thing

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#179

Post by esseph »

danielheath wrote: post #158
tempaccount5050 wrote: post #155

Earlier quoted context omitted.

You need the exact same people to run the infra in the cloud. If they don't have IT at all, they aren't spinning up cloud VMs. You're mixing together SaaS and actual cloud infra.

I'm one of those people, and I don't agree. Before I drop 5 figures on a single server, I'd like to have some confidence in the performance numbers I'm likely to see. I'd expect folk who are experienced with on-prem have a good intuition about this - after a decade of cloud-only work, I don't. Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. I've estimated…

> Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem.

Like what?

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#180

Post by PunchyHamster »

umvi wrote: post #134
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

Is using virtualization the only good way of taking a 288-core box and splitting it up into multiple parallel workloads? One time I rented a 384-core AMD EPYC baremetal VM in GCP and I could not for the life of me get parallelized workloads to scale just using baremetal linux. I wanted to run a bunch of CPU inference jobs in parallel (with each one getting 16 cores), but the scaling was atrocious - the more parallel…

They are used for VMs because the load is pretty spiky and usually not that memory heavy. For just running single app smaller core count but higher clocked ones are usually more optimal

>Anyway, I wasn't able to test using proxmox or vmware on there to split up cpu/memory resources; we decided instead to just buy a bunch of smaller-core-count AMD Ryzen 1Us instead, which scaled way better with my naive approac

If that was single 384 (192 times 2 for hyperthreading) CPU you are getting "only" 12 DDR5 channels, so one RAM channel is shared by 16c/32y

So just plain 16 core desktop Ryzen will have double memory bandwidth per core

Post reply on HN