Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AmpereOne: Cores Are the New MHz

jeffgeerling.com

Post reply on HN
Print view View original

21–30 of 132 posts

Re: AmpereOne: Cores Are the New MHz

#21

Post by stonemetal12 »

kevingadd wrote: post #16
amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

To be fair, utilizing 192 cores for a single process or operation is often exceedingly difficult. Scheduling and coordination and resource sharing are all really hard with thread counts that high, so you're probably best operating in terms of smaller clusters of 4-16 threads instead. Lots of algorithms stop scaling well around the 8-32 range.

When was the last time you saw a server with 10GB Ram no matter the number of cores\threads?

Re: AmpereOne: Cores Are the New MHz

#22

Post by chasil »

kevingadd wrote: post #16
amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

To be fair, utilizing 192 cores for a single process or operation is often exceedingly difficult. Scheduling and coordination and resource sharing are all really hard with thread counts that high, so you're probably best operating in terms of smaller clusters of 4-16 threads instead. Lots of algorithms stop scaling well around the 8-32 range.

I use xargs -P every weekend to back up my (Oracle) database.

It has 510 files that are around 2gb, and the parallel script uses rman to make a datafile copy on each one, lzip it, then scp it to my backup server.

I have xargs set to run 10 at once. Could I increase to 192? Yes.

Re: AmpereOne: Cores Are the New MHz

#23

Post by griomnib »

kevingadd wrote: post #16
amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

To be fair, utilizing 192 cores for a single process or operation is often exceedingly difficult. Scheduling and coordination and resource sharing are all really hard with thread counts that high, so you're probably best operating in terms of smaller clusters of 4-16 threads instead. Lots of algorithms stop scaling well around the 8-32 range.

You can also saturate your network link as well. I do like these improvements, but I’m old enough to know it’s always a game of “move the bottleneck”!

Re: AmpereOne: Cores Are the New MHz

#24

Post by WorkerBee28474 »

kevingadd wrote: post #16
amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

To be fair, utilizing 192 cores for a single process or operation is often exceedingly difficult. Scheduling and coordination and resource sharing are all really hard with thread counts that high, so you're probably best operating in terms of smaller clusters of 4-16 threads instead. Lots of algorithms stop scaling well around the 8-32 range.

> Lots of algorithms stop scaling well around the 8-32 range.

For the curious: https://en.wikipedia.org/wiki/Amdahl's_law

Re: AmpereOne: Cores Are the New MHz

#25

Post by sixothree »

amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

I'm not understanding how the 10 GB of RAM gets assigned to a core. Is this just his way to describe memory channels corresponding to the cores?

Re: AmpereOne: Cores Are the New MHz

#26

Post by wmf »

sixothree wrote: post #25
amelius wrote: post #14

> Cores are great, but it's all about how you slice them. Don't think of this as a single 192-core server. Think of it more like 48 dedicated 4-core servers in one box. And each of those servers has 10 gigs of high-speed RAM and consistent performance. Suddenly it sounds less impressive ...

I'm not understanding how the 10 GB of RAM gets assigned to a core. Is this just his way to describe memory channels corresponding to the cores?

It's not assigned at the hardware level (although Ampere has memory QoS) but you can assign RAM at the VM level which is what these CPUs are intended for.

Re: AmpereOne: Cores Are the New MHz

#28

Post by revnode »

loudmax wrote: post #20

I found this part particularly interesting: > Also, with 512 gigs of RAM and a massive CPU, it can run a 405 billion parameter Large Language Model. It's not fast, but it did run, giving me just under a token per second. If you're serious about running LLMs and you can afford it, you'll of course want GPUs. But this might be a relatively affordable way to run really huge models like Llama 405B on your own hardware. T…

It's really slow. Like, unusably slow. For those interested in self-hosting, this is a really good resource: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...

Re: AmpereOne: Cores Are the New MHz

#29

Post by Suppafly »

stonemetal12 wrote: post #21
kevingadd wrote: post #16

Earlier quoted context omitted.

To be fair, utilizing 192 cores for a single process or operation is often exceedingly difficult. Scheduling and coordination and resource sharing are all really hard with thread counts that high, so you're probably best operating in terms of smaller clusters of 4-16 threads instead. Lots of algorithms stop scaling well around the 8-32 range.

When was the last time you saw a server with 10GB Ram no matter the number of cores\threads?

>When was the last time you saw a server with 10GB Ram no matter the number of cores\threads?

Is that a lot or a little? I have a bunch that only have 8gb, it just depends on what they are being used for.

Post reply on HN