Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

81–90 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#81

Post by formerly_proven »

jfindley wrote: post #70
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

Do note though that AIUI these are all E-cores, have poor single-threaded performance and won't support things like AVX512. That is going to skew your performance testing a lot. Some workloads will be fine, but for many users that are actually USING the hardware they buy this is likely to be a problem. If that's you then the GraniteRapids AP platform that launched previously to this can hit similar numbers of threads…

E-cores aren't that slow, yesteryear ones were already around Skylake levels of performance (clock for clock). Now one might say that's a 10+ year old uarch, true, but those ten years were the slowest ten years in computing since the beginning of computing, at least as far as sequential programs are concerned.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#82

Post by hedora »

Tepix wrote: post #42
hedora wrote: post #38

So, they're selling this as an AI accelerator, with drop in compatibility with existing boards, and no boost to RAM bandwidth. As I understand things, it would be extremely unusual to ship a chip that was bound by floating point throughput, not uncached memory access, especially in the desktop/laptop space. I haven't been following the Intel server space too carefully, so it's an honest question: Was the old thing co…

No, they're not selling this as an "AI accelerator": Here is the quote: " The company says operators deploying 5G Advanced and future 6G networks increasingly rely on server CPUs for virtualized RAN and edge AI inference, as they do not want to re-architect their data centers in a bid to accommodate AI accelerators. " Edge AI usually means very small models that run fine on CPUs.

A very small model is going to be, what, 8GB? That'll easily blow through the caches. You're going to end up bottlenecked on DRAM either way.

So, I wonder if this is going to be any faster than the previous generation for edge AI.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#83

Post by user5994461 »

50lo wrote: post #27

With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU. I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

> I wonder whether the next bottleneck becomes software scheduling rather than silicon

Yep, the scheduling has been a problem for a while. There was an amazing article few years ago about how the Linux kernel was accidentally hardcoded to 8 cores, you can probably google and find it.

IMO the most interesting problem right now is the cache, you get a cache miss every time a task is moving core. Problem, with thousands of threads switching between hundreds of cores every few milliseconds, we're dangerously approaching the point where all the time is spent trashing and reloading the CPU cache.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#84

Post by zokier »

whateverboat wrote: post #34
50lo wrote: post #27

With packages like this (lots of cores, multi-chip packaging, lots of memory channels), the architecture is increasingly a small cluster on a package rather than a monolithic CPU. I wonder whether the next bottleneck becomes software scheduling rather than silicon - OS/runtimes weren’t really designed with hundreds of cores and complex interconnect topologies in mind.

I think linux can handle upto 1024 cores just fine.

afaik the mainline limit is 4096 threads. HP sells server with 32 sockets x 60 cores/socket x 2 threads/core = 3840 threads, so we are pretty close to that limit.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#85

Post by ozgrakkurt »

carefree-bob wrote: post #58
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

This factually did not play out like this in my experience.

The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway.

Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference.

Imo the cloud stuff is just too full of itself if you are trying to solve a problem that requires compute like hosting databases or similar. Just renting a machine from a provider like Hetzner and starting from there is the best option by far.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#86

Post by Imustaskforhelp »

hedora wrote: post #30
stego-tech wrote: post #18

Earlier quoted context omitted.

Yep, 100%, but that's why identifying compatible workloads first is key. A lot of orgs skip right to the savings pitch, ignorant of how their applications communicate with one another - and you hit the nail on the head that applications doing even some processing in a cloud provider will murder you on egress fees by trying to hybrid your app across them. Folks wanting one or the other miss savings had by effectively…

Any experience with the mid-to-small cloud providers that provide un-metered network ports and/or free interconnect with partner providers? (For various reasons, I just care about VPS/bare metal, and S3-compatiblity.) I'm looking at those because I'm having difficulty forecasting bandwidth usage, and the pessimistic scenarios seem to have me inside the acceptable use policies of the small providers while still predic…

Netcup and OVH provide free un-metered ports. There are actually lots of options available on the market. BuyVM is another good one.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#87

Post by user5994461 »

icegreentea2 wrote: post #69
user5994461 wrote: post #40

Earlier quoted context omitted.

Not competitive at all. It's easily visible on the laptop lines, where the same GPU manufactured on TSMC has 3 times the power/performance ratio compared to the Intel one. Putting more cores is just another desperate move to play the benchmark. Power is roughly quadratic with frequency, every time you fall behind competition, you can double the number of cores and reduce the frequency by 1.414 to compensate. Repeat a…

??? GPU vs CPU workloads are completely different. Comparing Panther Lake iGPU vs Ryzen iGPU is not going to tell you much about how high density server CPU performance will work out. The Panther Lake vs Ryzen laptop performance comparisons show that Pather Lake does well, basically trading against top end Ryzen AI laptop chips in both absolute performance, and performance per watt.

If you're not aware, Intel has released a lineup of laptops, with some models having the GPU made by them and some having the same GPU made by TSMC. That makes the comparison very direct. TSMC can deliver nearly 3 times the power/performance.

GPU and CPU manufacturing is the same thing, same node, same result. GPU is always maximizing perf/power ratio because it's embarrassingly parallel, leaving no room to game the benchmark. CPU can be gamed by having a single fast core, that drops performance in half as soon as you use another core.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#88

Post by nszceta »

carefree-bob wrote: post #58
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

Managing AWS is a ton of work anyway

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#89

Post by zer00eyz »

stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

> These sorts of core-density increases are how I win cloud debates in an org.

AMD has had these sorts of densities available for a minute.

> Identify the workloads that haven't scaled in a year.

I have done this math recently, and you need to stop cherry picking and move everything. And build a redundant data center to boot.

Compute is NOT the major issue for this sort of move:

Switching and bandwidth will be major costs. 400gb is a minimum for interconnects and for most orgs you are going to need at least that much bandwidth top of rack.

Storage remains problematic. You might be able to amortize compute over this time scale, but not storage. 5 years would be pushing it (depending on use). And data center storage at scale was expensive before the recent price spike. Spinning rust is viable for some tasks (backup) but will not cut it for others.

Human capital: Figuring out how to support the hardware you own is going to be far more expensive than you think. You need to expect failures and staff accordingly, that means resources who are going to be, for the most part, idle.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#90

Post by Drunkfoowl »

ozgrakkurt wrote: post #85
carefree-bob wrote: post #58

Earlier quoted context omitted.

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three ser…

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of…

[dead]
Post reply on HN