Does anyone understand this claim from the press release? > M4 Max supports up to 128GB of fast unified memory and up to 546GB/s of memory bandwidth, which is 4x the bandwidth of the latest AI PC chip. This allows developers to easily interact with large language models that have nearly 200 billion parameters. Having more memory bandwidth is not directly helpful in using larger LLM models. A 200B param model requires…
q4 or q5 quantization. Edit: Actually you'd want q3 to fit a 200B model into 128GB of RAM. e.g. this one is about 140GB https://huggingface.co/lmstudio-community/DeepSeek-V2.5-GGUF...
(Isn't that kind of like saying you can do real-time 4k encoding when you actually mean it can do real-time 720p encoding and then interpolate the missing pixels?)