Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformer architecture optimized for Apple Silicon

github.com

Post reply on HN
Print view View original

181–190 of 342 posts

Re: Transformer architecture optimized for Apple Silicon

#181

Post by epistasis »

fingerlocks wrote: post #177
epistasis wrote: post #130

Earlier quoted context omitted.

Me: Hey Siri, call Toyota. Siri: I'm sorry, that contact is not in your list Me: Siri, what is the number for Toyota. Siri: Toyota's phone number is 123-456-7890 [said too fast to remember or write down in one go] Me: Siri, call Toyota Siri: I'm sorry, you do not have that contact number. Me: &$@@&/&&/&!!!

Just tried this exact interaction. Siri gave me a list of nearest Toyotas and made the call. Seems to be working just fine. Starting to think all these Siri complaints are either made up or really outdated.

If you have the magic phrase to make it work, I'd like to know. I tried for about five minutes while driving yesterday, to no avail.

For me, Siri is like an 80s text adventure game, except I was better at those.

Edit: just now I was able to call by interacting a second time with Siri, and using the physical button. Using the button never would have occurred to me while driving.

Re: Transformer architecture optimized for Apple Silicon

#182

Post by johnthuss »

garblegarble wrote: post #129
qumpis wrote: post #111

Earlier quoted context omitted.

Offtopic, but for what purpose are you running llms locally (especially everyday)? My understanding was that the prompting requires to make them work at all was too great.

A little bit of research, a little bit of actual useful tasks - I'm interested in summarisation, which alpaca is decent at (even compared to existing summarisation-specific models I've tried) My other motivation is making sure I understand what offline LLMs can do... while I use GPT-3 and 4 extensively, I don't want to send something over the wire if I don't have to (e.g. if I can summarise e-mails locally, I'd rathe…

What prompt are you using for summarization? I’ve tried several variations without consistent results.

Re: Transformer architecture optimized for Apple Silicon

#183

Post by behnamoh »

nojito wrote: post #144
scarface74 wrote: post #94

Earlier quoted context omitted.

Apple gets $18B a year from Google to be the default search engine on Apple devices. Why would Apple build a search engine that would probably be inferior and not be able to monetize it?

They already have a search engine? Just swipe down on any iPhone.

I had to disable several features of iOS search due to ridiculously long delay and lag.

Re: Transformer architecture optimized for Apple Silicon

#184

Post by Tiktaalik »

BonoboIO wrote: post #107
arthurcolle wrote: post #84

Earlier quoted context omitted.

> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…

Same experience. Alexa is DIMENSIONS better than Siri. Siri can create a timer ... okay even an alarm. That’s it. It is comically bad. Their text to speech is excellent, but the rest is unbelievable bad. If Apple has some kind of silver bullet, it’s time to put it out or be left behind.

Amazon has publicly walked back from Alexa because turns out people don't use it for much beyond creating a timer and it doesn't make money.

I think it's quite likely that Apple realized the same quite a bit earlier and eased up on Siri versus focusing on other things.

Re: Transformer architecture optimized for Apple Silicon

#185

Post by oidar »

015a wrote: post #103
highwaylights wrote: post #70

Earlier quoted context omitted.

Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

They have every query their users enter into Spotlight. They have all the results it returns (who cares if they originated from Google, ok maybe they do, maybe not using it for anything is part of their deal with Google). They have your contacts, calendars, messages, email, music, workout history, tasks, real time location, which Siri already uses to e.g. recommend a destination when you get in your car to a degree o…

GPT-4 seems pretty "moatish".

Re: Transformer architecture optimized for Apple Silicon

#186

Post by theptip »

arthurcolle wrote: post #84
davnicwil wrote: post #19

Earlier quoted context omitted.

I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waitin…

> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give…

I don’t think Apple necessarily needs to lead the pack on LLM research. They just need to take the SOTA FOSS model and bake it into silicon, using their chip design expertise.

If you do that, you can do inference on mobile devices, which is a huge privacy win; it plus into their general privacy positioning in a big way. If you open up that SDK to developers it would be the iOS app gold rush all over again.

Note, Google is also moving in the direction of on-device inference with Coral, but they obviously will want to transmit the personalized model weights back to the mothership.

Re: Transformer architecture optimized for Apple Silicon

#187

Post by wetpaws »

davnicwil wrote: post #19
endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waitin…

Apple's exceptional hardware can only be rivaled by the absolute shitfest quality of their software. So no, I'm not buying it.

Re: Transformer architecture optimized for Apple Silicon

#188

Post by fnordpiglet »

bigDinosaur wrote: post #164
schrodinger wrote: post #93

Earlier quoted context omitted.

I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).

I was just setting reminders today and had Siri repeatedly fail to do what I want. The only use case I have for it is setting timers, which is does very well. I'd use it for much more if it was as good as ChatGPT.

This is entirely my point. They’re all crap at a fundamental level, none of them outperform any other. But the next generation, which will assuredly be backed by a LLM, will be shockingly powerful.

Re: Transformer architecture optimized for Apple Silicon

#189

Post by hobs »

015a wrote: post #103
highwaylights wrote: post #70

Earlier quoted context omitted.

Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

They have every query their users enter into Spotlight. They have all the results it returns (who cares if they originated from Google, ok maybe they do, maybe not using it for anything is part of their deal with Google). They have your contacts, calendars, messages, email, music, workout history, tasks, real time location, which Siri already uses to e.g. recommend a destination when you get in your car to a degree o…

Apple does not have the queries that their users enter into spotlight, They dont have all the results it returns.

You dont spawn a network request on every local search on an apple device.

Re: Transformer architecture optimized for Apple Silicon

#190

Post by smoldesu »

015a wrote: post #103
highwaylights wrote: post #70

Earlier quoted context omitted.

Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

They have every query their users enter into Spotlight. They have all the results it returns (who cares if they originated from Google, ok maybe they do, maybe not using it for anything is part of their deal with Google). They have your contacts, calendars, messages, email, music, workout history, tasks, real time location, which Siri already uses to e.g. recommend a destination when you get in your car to a degree o…

> And cost effective training and inference silicon is a moat, because its Physical, and like literally One Company on the planet makes it,

I got Nvidia on my paper. Did I do the math wrong?

Post reply on HN