It has A19 Pro. A19 Pro has matmul acceleration in its GPU, the equivalent of Nvidia's Tensor cores. This would make future Macs extremely viable for local LLMs. Currently, Macs have high memory bandwidth and high VRAM capacity but low prompt processing speeds. Give it a large context and it'll take forever before the first token is generated. If the M5 generation gets this GPU upgrade, which I don't see why not, the…
iPhone Air
611–620 of 1001 posts
Re: iPhone Air
#612Re: iPhone Air
#613Re: iPhone Air
#614It has A19 Pro. A19 Pro has matmul acceleration in its GPU, the equivalent of Nvidia's Tensor cores. This would make future Macs extremely viable for local LLMs. Currently, Macs have high memory bandwidth and high VRAM capacity but low prompt processing speeds. Give it a large context and it'll take forever before the first token is generated. If the M5 generation gets this GPU upgrade, which I don't see why not, the…
> If the M5 generation gets this GPU upgrade, which I don't see why not, then the era of viable local LLM inferencing is upon us. I don't think local LLMs will ever be a thing except for very specific use cases. Servers will always have way more compute power than edge nodes. As server power increases, people will expect more and more of the LLMs and edge node compute will stay irrelevant since their relative power w…
Mobile applications are also relevant. An LLM in your car could be used for local intelligence. I'm pretty sure self driving cars use some about of local AI already (although obviously not LLM, and I don't really know how much of their processing is local vs done on a server somewhere).
If models stop advancing at a fast clip, hardware will eventually become fast and cheap enough that running models locally isn't something we think about as being a non-sensical luxury, in the same way that we don't think that rendering graphics locally is a luxury even though remote rendering is possible.
Re: iPhone Air
#615I'm concerned about HN database storage capacity, so here's a simple way to think about this. If you're interested, consider buying it. If it's not for you, no need to argue a whole lot. Plenty of other topics worthy of discussion. XD
Re: iPhone Air
#616I want an extra thick model instead, let’s call it iPhone Travel (or Ultra?). Just thick enough so the cameras are no longer sticking out. Give me an all-week battery instead of an all-day one. Slim down the power usage and give a power saver mode that actually does make a difference. Let me go on a weekend trip in nature or festival without having to carry extra hardware or having to look for public charging station…
Personally, I really like being able to use lightweight MagSafe batteries instead of having a thicker iPhone. I used to agree with you, but the tech has gotten ridiculously good the last couple of years. With something like https://www.apple.com/shop/product/HRY02LL/A/anker-maggo-pow... , you get a magsafe battery that doubles the life of an iPhone and can be independently recharged, and is so slim that I can put it…
The wireless battery just slows the drain unless my phone is totally idle while charging. I really don't think wireless charging is very effective, at least it hasn't been with my 3yo phone and magnetic battery (even when both were new).
Re: iPhone Air
#617Can someone that is actually interested in this explain the appeal? Thin on its own I get but thin with a giant bump 100% defeats the whole point for me. Seems clear at this point there is little hope of them engineering their way into thin cameras.
Re: iPhone Air
#618It has A19 Pro. A19 Pro has matmul acceleration in its GPU, the equivalent of Nvidia's Tensor cores. This would make future Macs extremely viable for local LLMs. Currently, Macs have high memory bandwidth and high VRAM capacity but low prompt processing speeds. Give it a large context and it'll take forever before the first token is generated. If the M5 generation gets this GPU upgrade, which I don't see why not, the…
https://www.apple.com/watch/compare/?modelList=watch-series-...
In the past few weeks the oxymeter feature was enabled by a firmware update on series 10. Measurements are done on the watch, results are only reported on a phone.
Re: iPhone Air
#619That is the dumbest side profile I have ever seen. The camera bump and camera together are thicker than the rest of this thing. By its design it now demands a massive case or just won't ever sit even reasonably flat on a table. Ridiculous.
Re: iPhone Air
#620Can someone that is actually interested in this explain the appeal? Thin on its own I get but thin with a giant bump 100% defeats the whole point for me. Seems clear at this point there is little hope of them engineering their way into thin cameras.