Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

191–200 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#191

Post by Manuel_D »

Aurornis wrote: post #114
tgrowazay wrote: post #107

Earlier quoted context omitted.

Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, th…

$120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that…

> There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one.

You can still use cloud for excess capacity when needed. E.g. use on-prem for base load, and spin up cloud instances for peaks in load.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#192

Post by fc417fc802 »

hedora wrote: post #82
Tepix wrote: post #42

Earlier quoted context omitted.

No, they're not selling this as an "AI accelerator": Here is the quote: " The company says operators deploying 5G Advanced and future 6G networks increasingly rely on server CPUs for virtualized RAN and edge AI inference, as they do not want to re-architect their data centers in a bid to accommodate AI accelerators. " Edge AI usually means very small models that run fine on CPUs.

A very small model is going to be, what, 8GB? That'll easily blow through the caches. You're going to end up bottlenecked on DRAM either way. So, I wonder if this is going to be any faster than the previous generation for edge AI.

Perhaps instead of posting erroneous assertions to HN you could wander over to your LLM of choice and ask it something along the lines of: What are some examples of edge AI applications that achieve good performance on a CPU where memory bandwidth is severely limited compared to a GPU? Please link to publicly available models where possible.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#193

Post by rcxdude »

LunaSea wrote: post #91
ozgrakkurt wrote: post #85

Earlier quoted context omitted.

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of…

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

Moving around the physical hardware is a truly tiny part of the actual job, it's really not relevant. (especially nowadays, see the top level comment about how you can do an insane amount (probably more than the median cloud deployment) with a fraction of a rack).

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#194

Post by rcxdude »

carefree-bob wrote: post #80
boltzmann-brain wrote: post #73

Earlier quoted context omitted.

As opposed to talent to manage the AWS? Sorry, AWS loses here as well.

I know of AWS's reputation as a business and what the devs say who work there, so I have no argument against your point, except to say that they do manage to make it work. Somewhere in there must be some unsung heroes keeping the whole thing online.

The point being that AWS runs AWS, they don't run your business on AWS. You still need someone to actually set up AWS to do what you want, much like you would need someone to run your on-premises servers. And in my experience, the difference is not much.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#195

Post by foxglacier »

NoNameHaveI wrote: post #26

As a Yocto enthusiast, I am curious as to how much elapsed realtime would be needed for a clean Yocto build. Yocto is thread heavy, so with 288, it oughta be good.

As a fellow yocto enthusiast, I think they should call the process node 1.8e15 ym instead of the stupid legacy Angstrom unit.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#196

Post by overfeed »

NoNameHaveI wrote: post #26

As a Yocto enthusiast, I am curious as to how much elapsed realtime would be needed for a clean Yocto build. Yocto is thread heavy, so with 288, it oughta be good.

My Yocto build times on a 32-core AMD are negligible, <2 minutes for a full distro, IIRC. I suspect higher core counts have diminishing returns, especially since most dev builds are heavily cached.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#197

Post by Agingcoder »

fc417fc802 wrote: post #167
brcmthrowaway wrote: post #101

Earlier quoted context omitted.

Intel contributes to Linux, how is this a problem?

Wrong level of abstraction. NUMA is an additional layer. If the program (script, whatever) was written with a monolithic CPU in mind then the big picture logic won't account for the new details. The kernel can't magically add information it doesn't have (although it does try its best). Given current trends I think we're eventually going to be forced to adopt new programming paradigms. At some point it will probably m…

Yes exactly.

The kernel tries to guess as well as it can though - many years ago I hit a fun bug in the kernel scheduler that was triggered by numa process migration ie the kernel would move the processes to the core closest to the ram. It happened that in some cases the migrated processes never got scheduled and got stuck forever.

Disabling numa migration removed the problem. I figured out the issue because of the excellent ‘a decade of wasted cores’ paper which essentially said that on ‘big’ machines like ours funky things could happen scheduling wise so started looking at scheduling settings .

The main numa-pinning performance issue I was describing was different though, and like you said came from us needing to change the way the code was written to account for the distance to ram stick. Modern servers will usually let you choose from fully managed ( hope and pray , single zone ) to many zones, and the depending on what you’ve chosen to expose, use it in your code. As always, benchmark benchmarks.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#198

Post by phsau »

user5994461 wrote: post #87
icegreentea2 wrote: post #69

Earlier quoted context omitted.

??? GPU vs CPU workloads are completely different. Comparing Panther Lake iGPU vs Ryzen iGPU is not going to tell you much about how high density server CPU performance will work out. The Panther Lake vs Ryzen laptop performance comparisons show that Pather Lake does well, basically trading against top end Ryzen AI laptop chips in both absolute performance, and performance per watt.

If you're not aware, Intel has released a lineup of laptops, with some models having the GPU made by them and some having the same GPU made by TSMC. That makes the comparison very direct. TSMC can deliver nearly 3 times the power/performance. GPU and CPU manufacturing is the same thing, same node, same result. GPU is always maximizing perf/power ratio because it's embarrassingly parallel, leaving no room to game the…

Could you provide an article that explores this difference? I'd like to understand the mechanics of this and see how this conclusion is reached.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#199

Post by harrall »

throwup238 wrote: post #75
carefree-bob wrote: post #58

Earlier quoted context omitted.

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

> even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. This is really the core problem. Every time I’ve done the math on a sizable cloud vs on-prem deployment, there is so much money left on the table that the orgs can afford to pay FAANG-level salaries for several good SREs but never have we been able to find people to fill the roles or even kno…

You didn’t find people because SREs don’t do that.

You wanted sysadmins / IT / data center technicians.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#200

Post by b40d-48b2-979e »

lightedman wrote: post #112
LunaSea wrote: post #91

Earlier quoted context omitted.

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

"Those are not DevOps people." Real Devops people are competent from physical layer to software layer. Signed, Aerospace Devop

What a naïve take. Real™ DevOps know what they need to know.
Post reply on HN