Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

tomshardware.com

Post reply on HN
Print view View original

281–290 of 303 posts

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#281

Post by Aurornis »

formerly_proven wrote: post #121
Aurornis wrote: post #114

Earlier quoted context omitted.

$120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that…

> There's a second bus factor: What happens when that 8xH100 starts to get flakey? These come in a non-flakey variant?

You can tell in this thread who has and who hasn’t had to work with this hardware.

My favorite are the responses from people saying the warranty will have someone show up in “hours” and fix it. Best of luck to you.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#282

Post by ragall »

lightedman wrote: post #112
LunaSea wrote: post #91

Earlier quoted context omitted.

> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.

"Those are not DevOps people." Real Devops people are competent from physical layer to software layer. Signed, Aerospace Devop

There are no "Devops people". DevOps was created to mean a world where the DEVelopers are doing OPS, hence there cannot be "Devops people", as it would be a contradiction in terms. If you're specialized, you're just "Ops".

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#283

Post by Aurornis »

ragall wrote: post #279
stego-tech wrote: post #263

Earlier quoted context omitted.

> What universe are we living in that a single SRE can't manage even a single rack for less than half a million in total comp? The kind where TC isn’t measured by pod managed, but by person hired. Also the world where median rent in major metros is $3500 a month. If you think $120k is rich, you’re either operating in the boonies, outside the USA/Canada, or incredibly out of touch with the cost of living today and nee…

> outside the USA/Canada Indeed, there's no reason for a company to host this kind of batch compute in North America. You can get very good people in Eastern Europe at 1/3 the cost.

I like how this simple claim about being cheaper to self-host a single server has now escalated to opening an office in Eastern Europe and hiring people there to manage it.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#284

Post by ragall »

danielheath wrote: post #158
tempaccount5050 wrote: post #155

Earlier quoted context omitted.

You need the exact same people to run the infra in the cloud. If they don't have IT at all, they aren't spinning up cloud VMs. You're mixing together SaaS and actual cloud infra.

I'm one of those people, and I don't agree. Before I drop 5 figures on a single server, I'd like to have some confidence in the performance numbers I'm likely to see. I'd expect folk who are experienced with on-prem have a good intuition about this - after a decade of cloud-only work, I don't. Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. I've estimated…

> I've estimated our IT workload would roughly double if we were to add physically racking machines, replacing failed disks, monitoring backups/SMART errors etc.

That's why nowadays one would use a managed collocation service, not hosting a rack in the office basement.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#285

Post by ragall »

PaulKeeble wrote: post #113
stego-tech wrote: post #8

These sorts of core-density increases are how I win cloud debates in an org. * Identify the workloads that haven't scaled in a year. Your ERPs, your HRIS, your dev/stage/test environments, DBs, Microsoft estate, core infrastructure, etc. (EDIT, from zbentley: also identify any cross-system processing where data will transfer from the cloud back to your private estate to be excluded, so you don't get murdered with egr…

What has surprised me about the cloud is that the price has been towards ever increasing prices for cores. Yet the market direction is the opposite, what used to be a 1/2 or a 1/4 of a box is now 1/256 and its faster and yet the price on the cloud has gone ever up for that core. I think their business plan is to wipe out all the people who used to maintain the on premise machines and then they can continue to charge…

> What has surprised me about the cloud is that the price has been towards ever increasing prices for cores.

That makes a lot of sense. Cloud providers are selling compute, and as cores get faster, the single core gets more expensive.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#286

Post by __turbobrew__ »

esseph wrote: post #227
__turbobrew__ wrote: post #225

Earlier quoted context omitted.

BGP based routing is a major pain in the ass to do on-prem. If you want true HA in the datacenter you are going to need to utilize BGP.

I mean, BGP EVPN is the datacenter standard. (Linux infra / k8s / networking guy)

There are standards but actually designing a sane network architecture, buying all of the correct network hardware, and configuring all of the software to properly use that hardware is hard. At my company we have a team of about 20 people whose job it is to just design, install, and run the network.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#287

Post by ragall »

Aurornis wrote: post #283
ragall wrote: post #279

Earlier quoted context omitted.

> outside the USA/Canada Indeed, there's no reason for a company to host this kind of batch compute in North America. You can get very good people in Eastern Europe at 1/3 the cost.

I like how this simple claim about being cheaper to self-host a single server has now escalated to opening an office in Eastern Europe and hiring people there to manage it.

The trend of opening offices in Europe started one year into Covid. I'm sure that there are companies that haven't opened an office there yet, but fewer than one might imagine.

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#288

Post by esseph »

__turbobrew__ wrote: post #286
esseph wrote: post #227

Earlier quoted context omitted.

I mean, BGP EVPN is the datacenter standard. (Linux infra / k8s / networking guy)

There are standards but actually designing a sane network architecture, buying all of the correct network hardware, and configuring all of the software to properly use that hardware is hard. At my company we have a team of about 20 people whose job it is to just design, install, and run the network.

> There are standards but actually designing a sane network architecture, buying all of the correct network hardware, and configuring all of the software to properly use that hardware is hard. At my company we have a team of about 20 people whose job it is to just design, install, and run the network.

Network engineers do network engineering :)

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#289

Post by everforward »

PunchyHamster wrote: post #169
everforward wrote: post #156

Earlier quoted context omitted.

I work in SRE and the way you describe it would give me pause. The first is that SRE team size primarily scales with the number of applications and level of support. It does scale with hardware but sublinearly, where number of applications usually scales super linearly. It takes a ton less effort to manage 100 instances of a single app than 1 instance of 100 separate apps (presuming SRE has any support responsibiliti…

> The last is that I don’t know many SREs (maybe none at all) that are well-versed enough in all the hardware disciplines to manage a footprint the size we’re talking. If each SRE is 4 racks and a minimum team size is 4, that’s 16 racks. You’d need each SRE to be comfortable enough with networking, storage, operating system, compute scheduling (k8s, VMWare, etc) to manage each of those aspects for a 16 rack system. I…

I would call what you’re describing Datacenter Operations, with the exception of PXE boot.

You could have SRE do it, but most places don’t because you can get someone to swap a dead drive for way cheaper (it’s not really a complicated operation).

That growth of SRE teams comes from wanting reliability further up the stack. If you’re not on AWS, there’s no Aurora so someone has to be DBA to do backups, performance monitoring, configuring failovers for when a disk dies and RAID needs to rebuild, etc. Same for network, networked storage, yada yada

Re: Intel's make-or-break 18A process node debuts for data center with 288-core Xeon

#290

Post by everforward »

skissane wrote: post #214
everforward wrote: post #156

Earlier quoted context omitted.

I work in SRE and the way you describe it would give me pause. The first is that SRE team size primarily scales with the number of applications and level of support. It does scale with hardware but sublinearly, where number of applications usually scales super linearly. It takes a ton less effort to manage 100 instances of a single app than 1 instance of 100 separate apps (presuming SRE has any support responsibiliti…

> The first is that SRE team size primarily scales with the number of applications and level of support. It does scale with hardware but sublinearly, where number of applications usually scales super linearly. It takes a ton less effort to manage 100 instances of a single app than 1 instance of 100 separate apps (presuming SRE has any support responsibilities for the app). Talking purely in terms of hardware would ma…

Yeah, that’s valid, there are a few common layouts for SRE. I would call what you’re describing a horizontal layout (each team owns a layer for all apps that use that layer).

It sort of comes back to support levels. Your Infra SRE teams stay small if either a) an app SRE team owns application specific stuff, or b) SRE just doesn’t support application specific stuff. Eg if a particular query is slow but the DB is normal, who owns root causing that? Whoever does needs headcount, whether it’s app SRE, infra SRE or the devs.

Post reply on HN