761–770 of 1001 posts
Earlier quoted context omitted.
If the energy efficiency of things like Face ID was indeed so far so bad that you need a more efficient M3 Ultra, how come Face ID was integrated into smartphones years ago, apparently without significant negative impact on battery life?
You seem to be arguing with a strawman here -- who said you need an M3 Ultra for energy efficient Face ID?
Whoa. M3 instead of M4. I wonder if this was basically binning, but I thought that I had read somewhere that the interposer that enabled this for the M1 chips where not available. That Said, 512GB of unified ram with access to the NPU is absolutely a game changer. My guess is that Apple developed this chip for their internal AI efforts, and are now at the point where they are releasing it publicly for others to use.…
If Apple supported Linux (headless) natively, and we could rack m4 pros, I absolutely would use them in our Colo. The CPUs have zero competition in terms of speed, memory bandwidth. Still blown away no other company has been able to produce Arm server chips that can compete.
Earlier quoted context omitted.
That config would also use about 10x the power, and you still wouldn't be able to run a model over 32GB whereas the studio can easily cope with 70B llama and plenty of space to grow. I think it actually is perfect for local inference in a way that build or any other pc build in this price range would be.
The M3 Ultra studio also wouldn't be able to run path traced Cyberpunk at all no matter how much RAM it has. Workloads other than local inference LLMs exist, you know :) After all, if the only thing this was built to do was run LLMs then they wouldn't have bothered adding so many CPU cores or video engines. CPU cores (along with networking) being 2 of the specs highlighted by the person I was responding to, so they w…
Earlier quoted context omitted.
5,500 easily gets me either vastly more CPU cores if I care more about that or a vastly faster GPU if I care more about that. Or for both a 9950x + 5090 (assuming you can actually find one in stock) is ~$3000 for the pair + motherboard, leaving a solid $2500 for whatever amount of RAM, storage, and networking you desire. The M3 strikes a very particular middle ground for AI of lots of RAM but a significantly slower G…
That config would also use about 10x the power, and you still wouldn't be able to run a model over 32GB whereas the studio can easily cope with 70B llama and plenty of space to grow. I think it actually is perfect for local inference in a way that build or any other pc build in this price range would be.
Earlier quoted context omitted.
How would you compare the tok/sec between this setup and the M3 Max?
3.5 - 4.5 tokens/s on the $2,000 AMD Epyc setup. Deepseek 671b q4. The AMD Epyc build is severely bandwidth and compute constrained. ~40 tokens/s on M3 Ultra 512GB by my calculation.
1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance and cost , Gen 4 or Gen 5 have 8 or 9 t/s https://github.com/ggml-org/llama.cpp/discussions/11733 ) 2. AMD Epyc + 4090 or 5090 running KTransformers (over 10 t/s decode ? https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...)
Earlier quoted context omitted.
No native docker support, no headless management options (enterprise strength), Limited QoS management, lack of robust python support (out of the box), interactive user focused security model.
1. I run Docker and Podman on my Macs. 2. If you mean MDM, there are several good options. Screen sharing and SSH are build in. 3. In what sense? 4. `uv python install whatever` is infinitely better than upgrading on the OS vendor’s schedule. 5. What does that affect?
Is the Mac Pro dead or are they waiting for M4 Ultra refresh it?
Earlier quoted context omitted.
There isn't anything particularly high-bandwidth about Apple's DDR5 implementation, either. They just have a lot of channels, which is why I compared it to a 24-channel EPYC system. I agree that their integrated GPU architecture hits a unique design point that you don't get from nvidia, who prefer to ship smaller amounts of very different kinds of memory. Apple's architecture may be more suited to some workloads but…
M3 Ultra has 819GB/s, and a single epyc cpu with 12 channels has 460GB/s. As far as I know, llama.cpp and friends don’t scale across multiple sockets so you can’t use a dual socket Turin system to match the M3 Ultra. Also, 32GB DDR5 RDIMMS are ~200, so that’s 5K for 24 right there. Then you need 2x CPUs, at ~1K for the cheapest, and you need 2, and then a motherboard that’s another 1K. So for 8K (more, given you need…
Earlier quoted context omitted.
Given shared heritage, I would expect to see Apple work with FreeBSD before I would expect Apple to work with Linux.
I heard OpenBSD has been working for a while. I’m continually surprised Apple doesn’t just donate something like 0.1% of their software development budget to proton and the asahi projects. It’d give them a big chunk of the gaming and server markets pretty much overnight. I guess they’re too busy adding dark patterns that re-enable siri and apple intelligence instead.
Earlier quoted context omitted.
The Alpha architecture was 64-bit from the very beginning (though the amount of addressable virtual memory and physical memory depends on the processor implementation). I think it goes something like: - 2106x/EV4: 34-bit physical, 43-bit virtual - 21164/EV5: 40-bit physical, 43-bit virtual - 21264/EV6: 44-bit physical, 48-bit virtual The EV6 is a bit quirky as it is 43-bit by default, but can use 48-bits when I_CTL o…
My understanding is that the VAX from Digital was the mother of all "32-bit" architectures to replace the dead end PDP-11 (had a 64kbyte user space so wasn't really that much better than an Apple ][) and PDP-10/20 (36-bit words were awkward after the 8-bit byte took over the industry) The 68k and 386 protected mode were imitations of the VAX. Digital struggled with the microprocessor transition because they didn't wa…
VMS is the only OS (that I know of) that uses all 4 processor privilege modes.
Side note: The 21064 has such bizarre IPR mappings, the read values have lots of bits scrambled around compared to their write counterparts. This is likely a hardware design decision affecting the programmer's model, if I had to guess.