231–240 of 342 posts
As someone entirely at sea with the rapid pace of development in this sphere: 1. Is this a new LLM from Apple? 2. Is this a way to optimize running LLMs like Llama locally on M1 macs? 3. Something else altogether?
#2. A way to optimize running LLMs locally on Apple Silicon (including iPhones) I am just a little better informed. As I understand it, their code improves model performance and memory consumption using PyTorch and Huggingface libraries.
Earlier quoted context omitted.
> apple has shipped everything required for you to point at a light /in your house/ to turn it on or off (and optionally flick). this includes software - the individual components are built and ready - the only thing missing is gluing it together Could you expand on this bit? I’m pretty deep in the Apple ecosystem, but I’m not sure what you’re referencing here
Not OP, but I suspect this is a simple combination of AR and precise device location awareness. The former comes in the form of ARKit. I’m not aware of a framework for locating devices, but it wouldn’t surprise me if there were one.
Alternatively whole-home mapping via AR also solves this. No triangulation needed.
In 2 years, LLM similar to today’s capability of ChatGPT will be running locally on newer phones and macs.
Earlier quoted context omitted.
I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waitin…
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…
Earlier quoted context omitted.
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…
The part that Siri is bad at will be commoditized — someone will open-source a GPT-4-level language model. And Apple's moat will be being able to run that on-device with all the attendant benefits (privacy, zero marginal cost to the company, availability in more scenarios, etc)
The bottleneck with compute at the edge is (and will be) model size (both app download time and storage space on device). Stable Diffusion sits at about 2GB for fp16, Whisper Medium at 1.53GB, LLAMA is 120GB. Sure, Apple can ship an optimized model ( 1GB.
Yes, by the time iphones can smoothly run llama 30G, the state of the art gpt-x will probably be a terabyte. Skynet will forever live in the cloud with just assistant agents living on the devices.
Yeah, but if every individual is running decent chat-capable LLM, and businesses are running their own on their own devices, and those can communicated with each other, who needs to rely on Skynet?
Earlier quoted context omitted.
They have every query their users enter into Spotlight. They have all the results it returns (who cares if they originated from Google, ok maybe they do, maybe not using it for anything is part of their deal with Google). They have your contacts, calendars, messages, email, music, workout history, tasks, real time location, which Siri already uses to e.g. recommend a destination when you get in your car to a degree o…
> And cost effective training and inference silicon is a moat, because its Physical, and like literally One Company on the planet makes it, I got Nvidia on my paper. Did I do the math wrong?
Earlier quoted context omitted.
“hey siri, what’s the weather today?” “Now playing Eminem Love the way you lie featuring Rihanna”
Hey siri, turn the TV off. Which TV? Bedroom or Living Room or Everywhere? (Only 1 of the 2 TVs is ever on) I won’t spam this thread anymore, but I would be pleasantly surprised if it improved.
- Hey Siri, .
- I'm sorry, one of your devices is off
i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…
Isn't GPT so complex that it requires hundreds of GB of ram to be used? How's it going to run on iphone?