221–230 of 1001 posts
Earlier quoted context omitted.
Is there anything comparable that runs Linux, doesn't necessarily look as good, but is perhaps (a lot) cheaper/fixable? Or is this really pretty optimal? I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps? I want to get something for my company to run local models, wondering what would be a good option.
Asahi was stuck at M3 last time I checked it out.
I know, this is a bit of a meaningless comment, but it's funny in a way. Feels like late 90s again: - Xiaomi: We have matched Apple in CPU performance. - Apple: *Meep Meep...*
I do not find it funny tbh. I'm very very surprised that Xiaomi matches Apples speed even with the newest release, its not diminishing Xiaomis success.
Sick. Particularly stoked for the 10gb network card ($100 option) when using the Mac Mini as a server. Just wish the memory + NVMe prices could come back down to pre ai-goldrush prices. As $2999 for the M5 Pro with 64GB RAM feels painfully over-priced.
As someone who is a beginner at home networking and have a Mac Mini running as a Plex server at home; what is the use case for the 10gb network card?
The prices are ridiculous though. I may just keep rolling with my Windows 10 setup.
From what I’ve noticed, Apple products have been getting worse in quality year after year. Sometimes they even ruin their own devices with updates... I guess it’s all because of marketing.
Extraordinary claims require extraordinary evidence.
> M5 Ultra features a massive amount of high-bandwidth unified memory, up to 512GB, and delivers a staggering 1.2TB/s of unified memory bandwidth that is 50 percent higher than M3 Ultra. Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.
Is there anything comparable that runs Linux, doesn't necessarily look as good, but is perhaps (a lot) cheaper/fixable? Or is this really pretty optimal? I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps? I want to get something for my company to run local models, wondering what would be a good option.
Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels).
But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't.
However, macOS on M-series macs is excellent for local models. (Maybe not as excellent as a box full of the best nVidia GPUs, but still excellent).
So if you're getting Apple hardware, like Linux, and want to run all of it locally, a fine setup for a machine to run local models, with agentic characteristics:
- macOS running one of the many local model runners. I used to use Ollama and Whisper, and now use llama.cpp instead of Ollama. Others use LM Studio, oMLX, etc. Provide HTTP endpoints to access the models.
- Linux in a VM for overall control and orchestration, with standard VM settings, and bridged networking so it appears as its own machine on your network. Also, in here provide a robust shared file server for shared state. Use this VM as your desktop and primary access to the machine, if you like Linux.
- Linux in a VM to launch ephemeral, volatile containers, with the containers using a memory-only tmpfs overlay on top of a read-only Linux filesystem in a VM disk image, with tools in this filesystem. Alternatively, a writable Linux filesystem in a VM disk image, with disk buffering set to use macOS host buffering and discard fsync requests. These settings optimise for container disk performance for data that's only ephemeral which will be deleted soon or on system shutdown. (You can combined both VMs, but need to use two VM disks to get equivalent behaviour, and be careful about VM disk configuration of the two disks.)
- Containers spawned within that second Linux VM can be spawned very quickly and run quickly, so are ideal for LLM agents that need a quick sandbox. These sandboxes generally run faster than a macOS sandbox, despite being on the same machine with VM overhead, because Linux is faster at some things. Teach the LLMs to store files and memories they want to keep in the shared file server.
Apple still has the best hardware so I moved to it for the last few years, but the closed software ecosystem is terrible for taking advantage of it. I wasn't able to debug network errors (restartin my Mac worked), Metal was missing low level disassembly / debugging tools (there is some hard to use UI), but the worst thing was the inflexible windowing system. Even getting all the window handles on all screens/desktops…
I had a chance to try Omarchy past few days and it's just very different vs macOS. I honestly never had a problem with the windowing system ever since I built my own customization scripts (i.e. Hammerspoon). I can see the appeal for someone who wants ultimate customization though but macOS still wins overwhelmingly when it comes to polish, ecosystem, user experience, and apps (nothing comes close).
For example when using PyTorch I wanted to try to speed up my NN kernel by 2x by just using half precision and haven't noticed any speedup at all. Also I was missing the easy to use GNU tools that had to be mixed with Apple's tools.
I loved using Arc browser as well, and I'm missing it, but I guess I will do without it somehow (Chrome's vertical tabs are just not the same).
My main program missing from going back to Linux was ChatGPT Desktop, but now it's there.
I just checked out Hammerspoon, I'm happy for you that you wrote it, and looks great, but it has the same problem that I had: for security reasons Apple stopped allowing the window APIs to get all important information on other workspaces. You can only do it with Accessibility API. I was trying to fight with it but have up.
I know, this is a bit of a meaningless comment, but it's funny in a way. Feels like late 90s again: - Xiaomi: We have matched Apple in CPU performance. - Apple: *Meep Meep...*
I do not find it funny tbh. I'm very very surprised that Xiaomi matches Apples speed even with the newest release, its not diminishing Xiaomis success.
I enjoy it because of progress, not because of Apple.
Interesting times, to say the least!
Earlier quoted context omitted.
The DRAM market is cyclical. I don’t think anyone truly knows when, but it will happen. Fab capacity is being bought online; there’s just lead time. Noticeably greater intelligence is being achieved at the same number of parameters (see: Qwen3.8). I think the future will be bright, it might be a matter of time. And for tinkers, a used Epyc + DDR4 server can be great fun and epic value.
If it will happen in 100 years it will practically never happen (for us). Even 25 years would be a lot, it's half of a career. Could you provide more details about the Epyc + DDR4 server?
The above is a standard project management problem. We do this for lots of industry all the time. There is every reason to think you can get a new factory running in 5 years.
Note that I said 1 factory above. Some of the special machines we don't have the ability to make them fast enough to do 2 (I don't know the real number!) new factories in 5 years. Existing factories are using most of the special machine capacity to replace machines that wore out on the way - this can be corrected as well, but it adds another year and the expenses are much larger. Realistically though 1 new factory is likely enough.