Earlier quoted context omitted.
I'm not sure it's a death knell for frontier labs so much as a narrowing of what people need them for
When you've raised hundreds of billions in funding, every result except "to the moon" is a death knell.
Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
381–390 of 396 posts
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#382Earlier quoted context omitted.
> The M3 Ultra with 512GB was around $8k before Apple discontinued it The base model was $9k, that much RAM got you into $14k range.
512GB was around $9.5k. The $14k would be if you upgraded to 16TB SSD. https://youtu.be/jSYobH9kr1E?si=hc1xUQ37_SEbkDkj&t=1242
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#383Earlier quoted context omitted.
24 channels of DDR5-6400 is 1.2TB/s, M3 ultra is 0.8TB/s. They both use DDR5-6400. > a M5 studio would probably beat that performance for around half the cost. A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want. The price seems pretty comparable. The "problem" with doing this is actually that 128GB is too little memory, because you…
M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now. Even if this was M4 Ultra it would still be ~comparable to any EPYC system in bandwidth, but get to use the GPU for compute so potentially faster than the EPYC. Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably…
M3 Ultra uses a 1024-bit memory bus, which is a major inconvenience to Apple because they're soldering everything. In ordinary systems if you have 16 memory slots and any one of the memory chips is bad, you replace that stick. If the processor is bad, you replace the processor. If the system board is bad, you transfer the processors and memory to another one.
If any of those has a defect after you solder thousands of dollars worth of memory onto the same board as a >$1000 CPU, you're not doing well. Worse, the more memory chips you have and the more pins the CPU needs for its memory bus, the higher the chances of one of them having a defect.
In addition to that, when you get to that number of pins it starts getting harder to run the memory at the highest speeds. The M3 uses DDR5-6400 but some of the M5 line is using DDR5-9600. It may or may not be possible to do that speed when using a 1024-bit bus -- not every existing M5 even does it. If it isn't then the Ultra wouldn't be much faster than the Max since it would have to use a lower memory speed. If it is then the tolerances would have to be even tighter and increase the defect rate even more.
Which is to say, I can see why they haven't released an Ultra since the M3.
> but get to use the GPU for compute so potentially faster than the EPYC.
Only if the bottleneck is compute rather than memory bandwidth, and for LLMs it's generally memory bandwidth. And if something significant was compute bound, there are also higher core count CPUs.
> Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk
The M3 Ultra has a 480W TDP. There are relevant EPYC SKUs on SP5 at 125-200W/socket.
> you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth
The extra RAM allows you to fit the larger model in memory to begin with, without which it's pretty hopeless. The bottleneck typically is memory bandwidth for LLMs, but that's true pretty much regardless of the model size. Moreover, mixture of experts models require significantly more RAM for the same amount of compute/bandwidth.
> you also need at least one NVME disk
That's ~$100.
> As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200
The chips don't cost a materially different amount based on whether you solder them. Apple presumably discontinued the 256GB and 512GB versions of the M3 Ultra because they'd have had to add a similar number to the price.
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#384Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market. That means they're actually incentivized at least short term, to benefit PCs becoming strong enough to do local LLMs. Which makes this play make even more sense. Though, I've been saying for a while that the local AI inflectiom point is the death knell for these fron…
I do wish they have Xserve back, or a Mac Pro that is Rack Based and support multiple node with M6 Ultra. The Hyperscaler market is so large along with AI their old business case of Xserve didn't make sense no long hold true.
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#385Former AnandTech editor Gavin Bonshor had reports that the M7 would be manufactured on Intel's 18A node. https://bontechlabs.com/news/apple-is-reportedly-using-intel... Given the risks involved in establishing Apple Silicon designs with a new fab, I would expect early M7 parts to be in test production right now. The fundamental M7 design is already set in stone. Mark Gurman's Bloomberg article does not mention fabric…
I think that people are still underestimating the technical merits of Intel's 18A fabrication process. I haven't seen any competitor even try to address the backside power delivery of 18A. I suspect that Samsung,TSMC have something similar and doesn't talk about it. The design rules for the standard cell (sort of corresponding to the die area required by a transistor) for the Intel 18A seem to target dense, high perf…
They do, just not as hyped up as Intel. TSMC will have it after 20A. Either 18A or 14A. GAA was supposed to be in 3nm but didn't happen due to multiple reasons. So it is now delayed to 20A. Backside delivery was supposed to be 20A and also got pushed back as well.
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#386Earlier quoted context omitted.
If 8GB sticks of registered DDR5-6400 even exist, they're not common.
That was because memory used to be cheap. Will be curious if smaller capacities come back en vogue as a cost cutting mechanism.
And if you wanted that market segment then what you'd really do is produce a consumer GPU with 128GB of RAM.
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#387Earlier quoted context omitted.
> Hardware that can only run quantized models that take up GBs in VRAM That's the today hardware. Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output" and then releases M7 devices with enough memory and compute to run bigger models. > Unless Apple surprises everyone and brings back the XServe with M7, if n…
> Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output" It's revealing that they aren't doing this: no one wants to fund that gamble on the state of AI demand 12-18 months out, but ate happy to capitalize on their current product lines/capacity. > Desktops can dissipate more heat than the amount of electric…
They are doing this. All three of the DRAM manufacturers are actively expanding capacity. It's just not an instantaneous process.
> 100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
A desktop with the performance and the 160GB of VRAM of two H100s would sell like hotcakes. The power consumption is also largely proportional to the amount of compute rather than memory, and can be significantly improved by using lower clock speeds and then either making up for it with wider silicon or just not caring that it can run the same model but is e.g. 25% slower than hardware that uses twice as much electricity.
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#388Earlier quoted context omitted.
M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth. This has both a technical and human component. On the human side, top x86 execs refuse…
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. > teams from Intel and AMD absolutely HAD to know the truth. These people are professionals that acknowledge IPC is a stupid metric. If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads. x86 vendors recognized that they could…
Even if you think IPC is a stupid metric (it's not), ARM is still wiping the floor in total performance, performance per area, and performance per watt.
> If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
Fujitsu also makes a 512-bit SIMD that has very good performance, but that large SIMD eats area and power while giving very little back to consumers. Honestly, SME co-processors or 6-wide SVE are much closer to "space age" than simply making a wider packed SIMD.
> x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM.
AMD actually bought an architectural license and Intel used to have one too. They could (and should) undercut ARM entirely by making x86+RISC-V cores. You can talk about Intel's E-cores or AMD C-cores, but they still don't compare in the key metrics.
> AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
Which GPUs are you talking about? 4800U (Vega 8) was around 50% slower than M1's GPU. 5700G released in April 2021 and also had the same Vega 8 GPU design. They didn't see a bump until RDNA2 in Rembrandt in 2022 (by which point Apple was already on M2).
> nothing objectively prevents the x86 architecture from being power-efficient.
Citation needed. This is like saying nothing prevents branch delay slots from being good or VLIW/EPIC/Itanium is just as good as anything else. uop cache, extra pipeline stages for decoding, and memory ordering prediction are enough by themselves to use more power when under load. If you have actual proof that x86 workarounds have absolutely ZERO energy/area/latency, then show it.
> That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market.
Linus Torvalds pointed out that ARM server wouldn't take off until devs could run ARM natively on their laptops. Apple provided those laptops.
MS claims Cobalt 200 lowers carbon footprint by 40% vs x86 (which should mostly reflect power consumption per unit of work). ARM accounted for over a third of Azure deployments in 2025 and should increase as C200 becomes available.
> The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving
That hasn't been true for a long time now. Graviton5 in particular is a big deal. It removes the NUMA issues resulting in much lower latencies (and higher performance) for a lot of things. It gives 2.6x more cache per core and a general ~25% performance uplift (SAP said it was up to 60% for their workloads) which is more than the difference between graviton4 and Zen5 for most workloads. Graviton accounts for more than 50% of all new servers in AWS and basically all of their 1000 largest customers are using them.
> Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
Apple wasn't incapable -- they didn't want to because they aren't in the server market. That is the entire reason X-Elite/Nuvia exists. GPU stack is completely orthogonal to making server CPUs (you can make good CPUs without making good GPUs).
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#389Earlier quoted context omitted.
M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth. This has both a technical and human component. On the human side, top x86 execs refuse…
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. > teams from Intel and AMD absolutely HAD to know the truth. These people are professionals that acknowledge IPC is a stupid metric. If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads. x86 vendors recognized that they could…
At 30 watts TDP?
Re: Apple to skip high-end M6 Mac chips in favor of AI-focused M7 line
#390Earlier quoted context omitted.
I don’t think that represents the scenario at all, not to mention the fact that it’s literally not a planned economy (but also not very analogous to one, either). What’s really happening is that the effort of securing additional stock isn’t worth it because the price is so high that there aren’t enough buyers. If ground beef were to suddenly cost $50/pound, McDonald’s doesn’t raise the price of the Big Mac to $25 and…
Businesses are usually planned economies, and supply chain management is literal central planning. Apple failed to predict the demand for Mac Studios. Many other companies in its supply chain likely failed to predict that Apple would come back asking for more. There is no excess stock for some key components or the spare capacity to make them on demand. Apple would have to scour them from the market, likely paying mu…
It’s just not what those words mean.
Apple is a large company but they are still just one company.
A single business can’t be doing “central planning,” it is by definition not in charge of the whole market.
Central planning would be if the government mandated that Micron to make ## of memory chips and distribute ## of them to Company A and ## of them to Company B.
Apple has no such power.