771–780 of 1001 posts
-using an add-in video card for dom0 gpu
-apple's on-die gpu can be passed through to a qube
-apple's unified memory does not need OS X to adjust the RAM/VRAM wall
Since I doubt these caveats are in play, I think I may be stuck with enterprise hand-me-downs that I can pass through.
The 256gb and 512gb models have 8gb of ram. The 1tb and 2tb models have 16gb. Not a fan of tying ram to storage. https://www.apple.com/ipad-pro/specs/
The iPod classic had 160 GB of storage fifteen years ago. No device should be measuring storage in the gigabytes in 2024. Let alone starting at $1000 offering only 256GB. What ridiculousness.
I have years of photos and videos on there. Apparently the 256 GB model was the right choice for me?
Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…
I think these days everyone links their products with AI. Today even BP CEO linked his business with AI. Edge inference and cloud inference are not mutually exclusive choices. Any serious provider will provide both and the improvement in quality of services come from you giving more of your data to the service provider. Most people are totally fine with that and that will not change any time sooner. Privacy paranoia…
Seeing an M series chip launch first in an iPad must be result of some mad supply chain and manufacturing related hangovers from COVID. If the iPad had better software and could be considered a first class productivity machine then it would be less surprising but the one thing no one says about the iPads is “I wish this chip were faster”
Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…
> In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending data up to the cloud for processing). Their primary business goal is to sell hardware. Yes, they’ve diversified into services and being a shopping mall for all, but it is about selling lu…
Earlier quoted context omitted.
If these things will ever get MacOS support it will be useless with 8 GB of Ram. Such a waste of nice components.
I have a minimum of 64GB on my all my main developer machines (home, work, laptop), but I have a spare laptop with only 8GB of RAM for lightweight travel. Despite the entire internet telling me it would be "unusable" and a disaster and a complete disaster, it's actually 100% perfectly fine. I can run IDEs, Slack, Discord, Chrome, and do dev work without a problem. I can't run a lot of VMs or compile giant projects wi…
Earlier quoted context omitted.
I have a MacBook M2 and a PC with a 4090 ("just" one of them) - the VRAM barrier is usually what gets me with the 4090 when I try to run local LLMs (not train them). For a lot of things, my MacBook is fast enough, and with more RAM, I can run bigger models easily. And, it's portable and sips battery. The marketing hype is overblown, but for many (most? almost all?) people, the MacBook is a much more useful choice.
Expanding on this, I have an M2Pro (mini) & a tower w/GPU... but for daily driving the M2Pro idles at 15-35W whereas the tower idles at 160W. Under full throttle/load, even though the M2Pro is rated as less-performant, it is only using 105W — the tower/GPU are >450W!
Together with next-generation ML accelerators in the CPU, the high-performance GPU, and higher-bandwidth unified memory, the Neural Engine makes M4 an outrageously powerful chip for AI. In case it is not abundantly clear by now: Apple's AI strategy is to put inference (and longer term even learning) on edge devices. This is completely coherent with their privacy-first strategy (which would be at odds with sending dat…
> [Apple's] privacy-first strategy That is a marketing and advertising strategy.
- credit card required to install free apps such as the "private" Signal messenger
- location required just to show me the weather in a static place, lol
- claims to be e2e but apple controls all keys and identities
- basically sells out all users' icloud data in china, and totally doesn't do the same in the US, because tim pinky swears
- everything is closed source
Earlier quoted context omitted.
I think it will be a winning strategy. Lag is a real killer for LLMs. I think they'll have another LLM on a server (maybe a deal for openai/gemini) that the one on the device can use like ChatGPT uses plugins. But on device Apple have a gigantic advantage. Rabbit and Humane are good ideas humbled by shitty hardware that runs out of battery, gets too hot, has to connect to the internet to do literally anything. Apple…
> Lag is a real killer for LLMs. I'm curious to hear more about this. My experience has been that inference speeds are the #1 cause of delay by orders of magnitude, and I'd assume those won't go down substantially on edge devices because the cloud will be getting faster at approximately the same rate. Have people outside the US benchmarked OpenAI's response times and found network lag to be a substantial contributor…
groq is way better for inference speeds btw