Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformer architecture optimized for Apple Silicon

github.com

Post reply on HN
Print view View original

141–150 of 342 posts

Re: Transformer architecture optimized for Apple Silicon

#141

Post by philip1209 »

endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

I agree that you're going to see Apple start to push for on-device AI.

OpenAI promotes thin clients - run everything in the data center.

Apple has been a proponent of thick clients - run everything as possible locally.

So, it make sense that Apple will start to promote local LLM support. That's why they pushed Stable Diffusion support so quickly:

https://machinelearning.apple.com/research/stable-diffusion-...

Re: Transformer architecture optimized for Apple Silicon

#142

Post by LoganDark »

afavour wrote: post #99
parker_mountain wrote: post #88

Earlier quoted context omitted.

> how garbage Siri quality is today a few notes on this - siri is incredibly underinvested. They have another team that's building some sort of search and natural language processing engine, that has slowly sapped away some key headcount from the siri team. - apple doesn't get the full advantage of tons of user data from the wild. this is both a bug and a feature - the siri api model is clearly generations old, and i…

“Yes Siri is garbage but it doesn’t matter because Apple are probably working on a top secret new thing that will blow everyone else out of the water” You might be right but there’s no real evidence for it yet.

Hasn't Siri remained relatively unchanged for over a decade? I think the most recent innovation was just bringing it to Intel Macs.

Re: Transformer architecture optimized for Apple Silicon

#143

Post by btotes »

gitfan86 wrote: post #134
brokencode wrote: post #68

Earlier quoted context omitted.

Nvidia is also no David to Apple’s Goliath. They are both Goliaths who will most likely be battling to build the best hardware for LLMs in the near future.

NVDA has the potential to hit huge profit growth in the short term, but not long term

What makes you say that?

Re: Transformer architecture optimized for Apple Silicon

#144

Post by nojito »

scarface74 wrote: post #94
highwaylights wrote: post #70

Earlier quoted context omitted.

Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

Apple gets $18B a year from Google to be the default search engine on Apple devices. Why would Apple build a search engine that would probably be inferior and not be able to monetize it?

They already have a search engine?

Just swipe down on any iPhone.

Re: Transformer architecture optimized for Apple Silicon

#145

Post by revscat »

ngai_aku wrote: post #135
parker_mountain wrote: post #88

Earlier quoted context omitted.

> how garbage Siri quality is today a few notes on this - siri is incredibly underinvested. They have another team that's building some sort of search and natural language processing engine, that has slowly sapped away some key headcount from the siri team. - apple doesn't get the full advantage of tons of user data from the wild. this is both a bug and a feature - the siri api model is clearly generations old, and i…

> apple has shipped everything required for you to point at a light /in your house/ to turn it on or off (and optionally flick). this includes software - the individual components are built and ready - the only thing missing is gluing it together Could you expand on this bit? I’m pretty deep in the Apple ecosystem, but I’m not sure what you’re referencing here

Not OP, but I suspect this is a simple combination of AR and precise device location awareness. The former comes in the form of ARKit. I’m not aware of a framework for locating devices, but it wouldn’t surprise me if there were one.

Re: Transformer architecture optimized for Apple Silicon

#146

Post by yazaddaruvala »

scarface74 wrote: post #101
shmoogy wrote: post #31

Earlier quoted context omitted.

Gpt3.5-turbo runs laps around siri and is cheap, surely Apple will acquire talent to build something that does remote queries and falls back to local for the next 1-2 years while they figure out on device accelerated LLMs

People keep saying this. But it’s only half the problem to process natural language in text form. How well would ChatGPT do at understanding Spanish from a non native speaker with a southern US accent? (Raises hand). I’ve yet to see a speech to text system that does well at understanding speech by non native speakers or even native English speakers with a heavy regional accent.

We have yet to see any Large Speech Models or Large Multimodal Models that use speech and text as inputs.

It’s only the being but the general process to build this stuff now exists and the value proposition is crystal clear.

Re: Transformer architecture optimized for Apple Silicon

#147

Post by bjtitus »

lanza wrote: post #109
endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

OpenAI is big tech. Also, Apple has nowhere near the AI/ML staff that the rest of the tech giants have.

Big tech that already has comparable open source models that can run on consumer hardware?

Re: Transformer architecture optimized for Apple Silicon

#148

Post by crooked-v »

shmoogy wrote: post #31
endisneigh wrote: post #28

Earlier quoted context omitted.

I never said anything about openai not being relevant. the point is scale and good enough. gpt-4 is already good enough for most reasonable use cases, notably siri optimization. 5 years is a long time. 5 years ago llms didn't even exist.

Gpt3.5-turbo runs laps around siri and is cheap, surely Apple will acquire talent to build something that does remote queries and falls back to local for the next 1-2 years while they figure out on device accelerated LLMs

I don't think they'd even need to go that far to get started. Alpaca-LoRA is better than Siri when it comes to understanding queries and can already run on consumer GPUs. I would be really surprised if they couldn't get something equivalent running on the top-series A-chip devices in relatively short order.

Re: Transformer architecture optimized for Apple Silicon

#149

Post by doctoboggan »

endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

Apple is such a cautious company in many respects. Do you think they have the ability to ship a largely unpredictable piece of software like an LLM?

Re: Transformer architecture optimized for Apple Silicon

#150

Post by scarface74 »

nojito wrote: post #144
scarface74 wrote: post #94

Earlier quoted context omitted.

Apple gets $18B a year from Google to be the default search engine on Apple devices. Why would Apple build a search engine that would probably be inferior and not be able to monetize it?

They already have a search engine? Just swipe down on any iPhone.

Yes and at one point someone wrote a web front end for it. It’s laughably bad.
Post reply on HN