Why do these new machines have 192 cores, not a power of 2? (3* 2^6) = 192. The article mentions 256 and 512 cores coming. I get that various components can limit scaling like say bus speed or capacity. But 192, that's wacky.
AmpereOne: Cores Are the New MHz
71–80 of 132 posts
Re: AmpereOne: Cores Are the New MHz
#72Why do these new machines have 192 cores, not a power of 2? (3* 2^6) = 192. The article mentions 256 and 512 cores coming. I get that various components can limit scaling like say bus speed or capacity. But 192, that's wacky.
[flagged]
Re: AmpereOne: Cores Are the New MHz
#73Earlier quoted context omitted.
It's really slow. Like, unusably slow. For those interested in self-hosting, this is a really good resource: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...
You know, there's nothing wrong with running a slow LLM. For some people, they lack the resources to run an LLM on a GPU. For others, they want to try certain models without buying thousands of dollars of equipment just to try things out. Either way, I see too many people putting the proverbial horse before the cart: they buy a video card, then try to fit LLMs in to the limited VRAM they have, instead of playing arou…
> One token a second is worlds better than running nothing at all because someone told you that you shouldn't or can't because you don't have a fancy, expensive GPU.
It's a minuscule pittance, on hardware that costs as much as an AmpereOne.
Re: AmpereOne: Cores Are the New MHz
#74> I'm also happy to see systems like this that run without exotic water cooling and 240V power. Always funny to see 240V power described as exotic (or at least exotic-adjacent). It's the standard across practically the entire world except North and Central America!
They actually distribute 240V in USAmerica, one neutral wire and two opposite-phase 120V wires. Outlets in homes are 120V but wiring 240V is not difficult.
https://en.wikipedia.org/wiki/Split-phase_electric_power
This is different from Europe where we usually distribute 3 phases 130V.
Re: AmpereOne: Cores Are the New MHz
#75> I'm also happy to see systems like this that run without exotic water cooling and 240V power. Always funny to see 240V power described as exotic (or at least exotic-adjacent). It's the standard across practically the entire world except North and Central America!
Every home in the US with an electric stove or water heater is using 240v electricity. It isn't really very exotic. 3 phase power to homes IS very exotic though and I'm not sure why it would be useful in homes.
3 phase power is near universal inncentral and northern europe.
And it's useful for powering saunas, electric warm water or bigger tools
Re: AmpereOne: Cores Are the New MHz
#76Why do these new machines have 192 cores, not a power of 2? (3* 2^6) = 192. The article mentions 256 and 512 cores coming. I get that various components can limit scaling like say bus speed or capacity. But 192, that's wacky.
Don't ask GPT:
> Architectural design: The physical layout of the chip may favor certain core arrangements. For example, a 48-core processor could be designed as a 6x8 grid, which efficiently utilizes the square shape of the silicon die6.[1]
Major Deep Thought vibes.
[1] https://www.perplexity.ai/search/why-would-a-cpu-have-48-cor...
Re: AmpereOne: Cores Are the New MHz
#77I found this part particularly interesting: > Also, with 512 gigs of RAM and a massive CPU, it can run a 405 billion parameter Large Language Model. It's not fast, but it did run, giving me just under a token per second. If you're serious about running LLMs and you can afford it, you'll of course want GPUs. But this might be a relatively affordable way to run really huge models like Llama 405B on your own hardware. T…
It's really slow. Like, unusably slow. For those interested in self-hosting, this is a really good resource: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...
> this is a really good resource: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...
Yes, it is.
But it has not been updated for seven months. Do things change so slowly?
Re: AmpereOne: Cores Are the New MHz
#78Earlier quoted context omitted.
Every home in the US with an electric stove or water heater is using 240v electricity. It isn't really very exotic. 3 phase power to homes IS very exotic though and I'm not sure why it would be useful in homes.
3 phase power is near universal inncentral and northern europe. And it's useful for powering saunas, electric warm water or bigger tools
why? The benefits of 3-phase power are really only used by factories with lots of large electric motors. 3 phase has no benefit for resistance heating.
Re: AmpereOne: Cores Are the New MHz
#79Really a pity that Oracle killed off SPARC. They already had 32-core CPUs almost a decade ago but Oracle never really understood the value that SPARC and Solaris brought to the table.
I was a huge Sun fan for most of the 90's and had several Sun systems at home. By the early 2000's, Sun was no longer competitive. Between Intel and Linux, the writing was on the wall. I haven't seen a Sun system used in production since 2003.
They are still being used in telecom. Not the new SPARC M7 series, but what was shipping in 2004-2005 timeframe is still in the rack and doing stuff (which is pretty crazy).
Re: AmpereOne: Cores Are the New MHz
#80Earlier quoted context omitted.
I've run plenty of microservices with 256 or 512 GB of RAM, and they were handling large loads. So long as each request is short lived, and using a runtime with low per request overhead (e.g. Node), memory is not really a problem for many types of workloads.
Just to be clear, you meant MB of RAM, right?
Oops, yeah. MB.