For local llm bad
MacBook Pro with M5 Pro and M5 Max
821–830 of 1001 posts
Re: MacBook Pro with M5 Pro and M5 Max
#822Earlier quoted context omitted.
My 32gb m1 max was probably the best purchase I've made. Still plenty of headroom in performance left in this beast. Wonder what reason they'll use to end software support in the future. Bet it'll be some security hardware they make up for the sake of forcing upgrades.
my tinfoil hat theory is that they make small features depend on new hardware. for example, let's say the new os depends on m5's exclusive thumbnail generator accelerator, and let's say it improves speed by a 20%. now, your M1 notebook than on previous OSes uses standard gpu acceleration for thumbnails will not have this specialized hardware acceleration, it will have software fallback that will be 90% slower. you wo…
>my tinfoil hat theory is that they make small features depend on new hardware.
The general case is hardly a "tinfoil hat theory". They openly do that, and the major reason is to tie to new hardware adoption.
That said, it doesn't usually work like you call it. It's not adding new features depending on HW optimization to slow older machines down (after all one could just not use those features in an older machine, or toggle them off).
It's rather: you want to get these shiny new features, which is all we advertise for iOS/macOS N+1, and the main new changes? The big ones will only work if you have a newer machine, even though we could trivially enable them on older machines (and some don't even need special hardware, as there are third-party hacks that unlock them and they work fine).
Re: MacBook Pro with M5 Pro and M5 Max
#823Earlier quoted context omitted.
They bought the fab time for that RAM 2-3 years ago. Apple is renowned for their foresight and preparation. We'll eventually see price increases from Apple's RAM upgrade, but we're not there yet.
Commodity futures made sense to me at FedEx- they would pay money with a supplier for the option to buy gas/oil at X price at Y date in the future. It costs more than just agreeing to pay for it at that price in the future, but if deliveries went way down (or prices) it'd be less costly to "back out". I wonder if there's a fab time secondary market where Wall Street types are making millions off speculating fab time.
Hm I don't think a secondary market would work very well, using fab time productively requires lots of knowledge and collaboration with the provider. Compared to resources like grain or oil where it's basically "just come and pick it up when it's there".
Re: MacBook Pro with M5 Pro and M5 Max
#824Earlier quoted context omitted.
Mine still runs like the first day I had it. There's basically nothing that is limiting me with the machine as it is, everything is just me being slow to code. I don't see why I need a new computer at the moment. In the past, I always got to a stage where the machine felt sluggish.
Yeah my M1 is still insanely snappy. Would be nice to have some extra legroom for things like compilation, but I'm far from feeling this device isn't sufficient for me.
[deleted]
Re: MacBook Pro with M5 Pro and M5 Max
#825Great, I have a MacBook Pro M5 for work which i know IS really fast because the whole suite of specs run in les than two minutes, but except for that my "old" air m1 with 16g of ram feels super snappy.
I am just excited waiting future releases (in 5 or 10 years) able to run local llms for coding.
Re: MacBook Pro with M5 Pro and M5 Max
#826Earlier quoted context omitted.
You barely see any liquid glass on Tahoe. I keep my dock hidden and it's just the icons mostly which aren't that different than before.
Same here. Not really understanding the complaints for macOS. I think the addition of icons in the context and menus is worse than glass.
The border radius are terrible
Re: MacBook Pro with M5 Pro and M5 Max
#827A £3000 laptop doesn't come with a fucking charger. It's £99 extra for the one that came with the m4's. thanks a lot Apple, the enshitification is real!
Only in the EU/UK
Re: MacBook Pro with M5 Pro and M5 Max
#828A £3000 laptop doesn't come with a fucking charger. It's £99 extra for the one that came with the m4's. thanks a lot Apple, the enshitification is real!
This isn't enshitification. Most people have plenty of chargers. Its a good thing you can opt in/out.
Re: MacBook Pro with M5 Pro and M5 Max
#829Earlier quoted context omitted.
They bought the fab time for that RAM 2-3 years ago. Apple is renowned for their foresight and preparation. We'll eventually see price increases from Apple's RAM upgrade, but we're not there yet.
Commodity futures made sense to me at FedEx- they would pay money with a supplier for the option to buy gas/oil at X price at Y date in the future. It costs more than just agreeing to pay for it at that price in the future, but if deliveries went way down (or prices) it'd be less costly to "back out". I wonder if there's a fab time secondary market where Wall Street types are making millions off speculating fab time.
There were/are DRAM futures markets based on this hypothesis, historically they never worked very well because DRAM prices (at least for the same size/speed) have moved downwards so consistently over decades. That is until <6 months ago.
Re: MacBook Pro with M5 Pro and M5 Max
#830On M4 Max 128GB we're seeing ~100 tok/s generation on a 30B parameter model in our from scratch inference engine. Very curious what the "4x faster LLM prompt processing" translates to in practice. Smallish, local 30B-70B inference is genuinely usable territory for real dev workflows, not just demos. Will require staying plugged in though.
The memory bandwith on M4 Max is 546 GB/s, M5 Max is 614GB/s, so not a huge jump. The new tensor cores, sorry, "Neural Accelerator" only really help with prompt preprocessing aka prefill, and not with token generation. Token generation is memory bound. Hopefully the Ultra version (if it exists) has a bigger jump in memory bandwidth and maximum RAM.
I noticed that even on my M3 MLX tends to do prefill it a lot faster than llama.cpp and GGML models. Anyone knows how they do it?