In 2 years, LLM similar to today’s capability of ChatGPT will be running locally on newer phones and macs.
Transformer architecture optimized for Apple Silicon
91–100 of 342 posts
Re: Transformer architecture optimized for Apple Silicon
#92Earlier quoted context omitted.
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…
The part that Siri is bad at will be commoditized — someone will open-source a GPT-4-level language model. And Apple's moat will be being able to run that on-device with all the attendant benefits (privacy, zero marginal cost to the company, availability in more scenarios, etc)
yeah, makes sense. At least these integrations at a lower level are happening now. I just can't help but feel disappointed that we didn't see earlier cohesion at least a few years ago, maybe around the early or pre-M1 era when there were all those A7, A8 whatever SoCs. But in retrospect, that is basically 10 minutes ago in the history of the universe so yeah, I'm just overly excitable!
Re: Transformer architecture optimized for Apple Silicon
#93Earlier quoted context omitted.
I think this is wrong because you can always build a search engine - it’s not hard - and you can supplement the learned models with cryptographically secured private data to build a private personalized model. In fact this seems in all ways superior to any alternative, and strikes at the heart of the Google spy ware ad spam business model. And seriously, did you really try to assert the Google whatever it is has outp…
> And seriously, did you really try to assert the Google whatever it is has outperformed Siri? Anything outperforms Siri. Google Assistant is head and shoulders above Alexa and Siri.
I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).
Re: Transformer architecture optimized for Apple Silicon
#94Earlier quoted context omitted.
I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waitin…
Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.
Apple gets $18B a year from Google to be the default search engine on Apple devices. Why would Apple build a search engine that would probably be inferior and not be able to monetize it?
Re: Transformer architecture optimized for Apple Silicon
#95i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…
I’ll give it 2 years max to reach ChatGPT like capabilities running locally on latest iPhones and Macs.
Lets hope Apple doesn’t screw this up.
Re: Transformer architecture optimized for Apple Silicon
#96Earlier quoted context omitted.
“Hey siri, what is today’s date?” “Sorry, I’m having trouble connecting to the network”
“hey siri, what’s the weather today?” “Now playing Eminem Love the way you lie featuring Rihanna”
And that has nothing to do with an LLM. That has to do with speech to text algorithms. It also doesn’t take advanced AI to do a simple intents based answers.
Re: Transformer architecture optimized for Apple Silicon
#97In 2 years, LLM similar to today’s capability of ChatGPT will be running locally on newer phones and macs.
[deleted]
Re: Transformer architecture optimized for Apple Silicon
#98i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…
Apple will never have their tech run locally for AI anymore than they allow Siri to search the internet offline or while driving
How would anyone search the internet offline?
Re: Transformer architecture optimized for Apple Silicon
#99Earlier quoted context omitted.
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…
> how garbage Siri quality is today a few notes on this - siri is incredibly underinvested. They have another team that's building some sort of search and natural language processing engine, that has slowly sapped away some key headcount from the siri team. - apple doesn't get the full advantage of tons of user data from the wild. this is both a bug and a feature - the siri api model is clearly generations old, and i…
“Yes Siri is garbage but it doesn’t matter because Apple are probably working on a top secret new thing that will blow everyone else out of the water”
You might be right but there’s no real evidence for it yet.
Re: Transformer architecture optimized for Apple Silicon
#100i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…
I'm expecting the Mac Pro to have AI/ML massive GPU modules and/or specialized ML dedicated compute available as add-ons for in-house training. For a not-so-small fee of course. They have all the pieces, it just needs to be brought to bear on the ML training market.
UMA is the secret.
A 256GB Mac Pro can dedicate almost all of that to AI.
Their GPUs are the weak spot. The Nvidia 4080 is 2.5x faster than the M2, and the A100 is 15 times faster.