Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

111–120 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#111

Post by ozgrakkurt »

LunaSea wrote: post #91
ozgrakkurt wrote: post #85

Earlier quoted context omitted.

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of…

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

To be clear, I'm not writing about on-premise. I mean difference between managed cloud and renting dedicated servers

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#112

Post by lightedman »

LunaSea wrote: post #91
ozgrakkurt wrote: post #85

Earlier quoted context omitted.

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of…

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

"Those are not DevOps people."

Real Devops people are competent from physical layer to software layer.

Signed,

Aerospace Devop

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#113

Post by PaulKeeble »

stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

What has surprised me about the cloud is that the price has been towards ever increasing prices for cores. Yet the market direction is the opposite, what used to be a 1/2 or a 1/4 of a box is now 1/256 and its faster and yet the price on the cloud has gone ever up for that core. I think their business plan is to wipe out all the people who used to maintain the on premise machines and then they can continue to charge similar prices for something that is only getting cheaper.

Its hard drive and SSD space prices that stagger me on the cloud. Where one of the server CPUs might only be about 2x the price of buy a CPU for a few years if you buy less in a small system (all be it with less clock speed usually on the cloud) the drive space is at least 10-100x the price of doing it locally. Its got a bit more potential redudency but for that overhead you can repeat that data a lot of times.

As time has gone on the deal of cloud has got worse as the hardware got more cores.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#114

Post by Aurornis »

tgrowazay wrote: post #107
throwup238 wrote: post #75

Earlier quoted context omitted.

> even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. This is really the core problem. Every time I’ve done the math on a sizable cloud vs on-prem deployment, there is so much money left on the table that the orgs can afford to pay FAANG-level salaries for several good SREs but never have we been able to find people to fill the roles or even kno…

Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, th…

$120K isn't going to cover the fully loaded costs of an SRE who can set up and run that.

Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever.

If there's an issue with the server while they're sick or on vacation, you just stop and wait.

If they take a new job, you need to find someone to take over or very quickly hire a replacement.

There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one. You can start diagnosing things and replacing parts and hope it gets to the root issue, but that's more downtime.

Going on-prem like this is highly risky. It works well until the hardware starts developing problems or the person in charge gets a new job. The weeks and months lost to dealing with the server start to become a problem. The SRE team starts to get tired of having to do all of their work on weekends because they can't block active use during the week. Teams start complaining that they need to use cloud to keep their project moving forward.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#115

Post by rishabhaiover »

50lo wrote: post #27

With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU. I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

That's a great point. Linux has introduced io_uring, and I believe that gives us the native primitives to hide latency better?

But that's just one piece of the puzzle, I guess.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#116

Post by highfrequency »

carefree-bob wrote: post #58
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

Given how good Apple Silicon is these days, why not just buy a spec'd out Mac Studio (or a few) for $15k (512 GB RAM, 8 TB NVMe), maybe pay for S3 only to sync data across machines. No talent required to manage the gear. AWS EC2 costs for similar hardware would net out in something ridiculous like 4 months.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#117

Post by rishabhaiover »

jeffbee wrote: post #60
50lo wrote: post #27

With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU. I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

There definitely are bottlenecks. The one I always think of is the kernel's networking stack. There's no sense in using the kernel TCP stack when you have hundreds of independent workloads. That doesn't make any more sense than it would have made 20 years ago to have an external TCP appliance at the top of your rack. Userspace protocol stacks win.

io_uring?

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#118

Post by justsomehnguy »

carefree-bob wrote: post #58
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

> main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear

Not quite. If you hire a bad talent to manage your 'cloud gear' then you would find what the mistakes which would cost you nothing on-premises would cost you in the cloud. Sometimes - a lot.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#119

Post by bgnn »

rubyn00bie wrote: post #6

I’ve not kept up with Intel in a while, but one thing that stood out to me is these are all E cores— meaning no hyperthreading. Is something like this competitive, or preferred, in certain applications? Also does anyone know if there have been any benchmarks against AMDs 192 core Epyc CPU?

In HPC, like physics simulation, they are preferred. There's almost no benefit of HT. What's also preferred is high cluck frequencies. These high core count CPUs nerd their clixk frequencies though.
Post reply on HN