Leaked Apple meeting shows how dire the Siri situation is

Whatever Tesla uses for its voice navigation just works every time. No idea why Siri (and Alexa) still have such poor command recognition by comparison.

I'm not as familiar with Shortcuts API but from a quick glance it seems less rich than apple events/apple script. With LLM + apple script you could achieve computer use agents on easy mode. Not just one-off "what's the weather" queries but complex multi-step interactions like "send the latest photo I took to John".

To start with, Automator on mac would be the perfect place for LLM integration. And Script Editor too. Being perhaps one of the few read-only languages, people would probably _prefer_ an LLM spit out applescript for them. And Apple probably has the highest quality data set internally. Combined with the fact that there there is a uniform way to specify the API surface (sdef), this is a task that is possible by most LLMs today. Just apply a little marketing spin to change the angle from "niche power user feature" to "apple uses computer-use agent AI to allow average joe to automate their entire workflow" and it's a smash hit.

From there it's not much of a stretch to slap some speech recognition (just take Whisper, it already beats whatever iOS uses), add some guardrails, and have it be orders of magnitude better than what Siri currently is capable of. And this is all possible with today's LLMs, and thanks to deepseek without paying a cent to anyone else. When interactive computer-use does get mostly solved, that can be added in as a fallback for anything where applescript doesn't cut it (e.g. web navigation, electron apps, etc.). But there's a clear logical progression that allows you to easily evolve the user experience as the technology matures without having to throw out your stack the entire time.

But to me I think their fate was sealed was Shortcuts was shipped on mac when Automator already existed. And it's clear apple events has been a languishing feature, with integration in native apps already breaking.

LLMs are probably better at "understanding" what the user wants, e.g. "Hey Siri, I have to take my kids to the doctor tomorrow" would reply "Do you want me to make a calendar event?".

Asking ChatGPT, it even has suggestions like "Do you want me to check traffic, do you want me to make a checklist for what to bring?".

If I continue with "but I can't make it", one of the suggestions ChatGPT imagine possible is: "Would you like me to send a message to your partner, babysitter, or anyone else involved?"

Enterprise MDM and Apple Configurator can disable voice assistant entirely, which may be needed in some work contexts.

The same year the tech industry got taken by storm by LLMs, Apple announced a brutally expensive, niche VR/AR headset with no particular use case. That product was essentially DOA.

I think this is the first time in decades where Apple took a step from their back foot. The Vision Pro announcement during the initial hype cycle of ChatGPT made it look like they were just totally out of step with the industry. It was a dud product that cost billions in R&D, the market for it was unclear, and the industry zeitgeist (and talent) was going in a totally different direction.

So their response was to rush something out the door to avoid looking like they were caught flat-footed. Now it's abundantly obvious they were unprepared for AI, they've failed to ship features they promised, and they have to commit resources to support an additional platform that no one cares about because they did ship that one. Worst of all worlds.

Had they slept on the Vision Pro, I'm pretty convinced they could have credibly kept their powder dry on LLMs, like they've done with many industry hype cycles in the last. People would've assumed they were doing the Apple thing of not being first to ship something, but the first to ship compelling.