Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformer architecture optimized for Apple Silicon

github.com

Post reply on HN
Print view View original

161–170 of 342 posts

Re: Transformer architecture optimized for Apple Silicon

#161

Post by alwayslikethis »

abraxas wrote: post #151
endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

Greed is a fabulous motivator but it can't surpass the laws of physics. This is probably why we don't have a cure for cancer or room temperature superconductors or jurassic parks. It may be so that scaling down a language model like GPT 4 is not possible on hardware systems orders of magnitude smaller than the one used by OpenAI. I'm not saying it's impossible but it's fallacious to just assume outright that it's ine…

There is nothing impossible about a Jurassic park in terms of laws of physics though. It's a matter of getting the right materials to recreate dinosaurs.

Same goes for cancer cures. I'm not sure if there are physical limitations preventing a room temp superconductor.

Re: Transformer architecture optimized for Apple Silicon

#162

Post by bigDinosaur »

LoganDark wrote: post #142
afavour wrote: post #99

Earlier quoted context omitted.

“Yes Siri is garbage but it doesn’t matter because Apple are probably working on a top secret new thing that will blow everyone else out of the water” You might be right but there’s no real evidence for it yet.

Hasn't Siri remained relatively unchanged for over a decade? I think the most recent innovation was just bringing it to Intel Macs.

It has had features removed.

Re: Transformer architecture optimized for Apple Silicon

#163

Post by dragonwriter »

brokencode wrote: post #68
ibejoeb wrote: post #45

Earlier quoted context omitted.

OpenAI's relationship with Microsoft pretty much makes it big tech. I think you have a point as long Apple keeps innovating its hardware, but it's not really a David vs Goliath situation.

Nvidia is also no David to Apple’s Goliath. They are both Goliaths who will most likely be battling to build the best hardware for LLMs in the near future.

Google is also in the machine learning hardware, game, on both the remote end and the device end.

Re: Transformer architecture optimized for Apple Silicon

#164

Post by bigDinosaur »

schrodinger wrote: post #93
coolspot wrote: post #80

Earlier quoted context omitted.

> And seriously, did you really try to assert the Google whatever it is has outperformed Siri? Anything outperforms Siri. Google Assistant is head and shoulders above Alexa and Siri.

I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).

I was just setting reminders today and had Siri repeatedly fail to do what I want. The only use case I have for it is setting timers, which is does very well. I'd use it for much more if it was as good as ChatGPT.

Re: Transformer architecture optimized for Apple Silicon

#165

Post by astrange »

month13 wrote: post #56
s1k3s wrote: post #47

Earlier quoted context omitted.

Wait, so there's a way to make a model as smart as GPT but with less parameters? Isn't that why it's so good?

This is an older paper, but DeepMind alleges in their Chinchilla paper that far better performance can be extracted with fewer parameters; quote "We find that current large language models are significantly under-trained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant." It's difficult to evaluate a LLM's performance as it's all qualitative, but Meta's…

Chinchilla is aimed at finding a cost-performance tradeoff as well, not the optimal amount of training. If cost is no barrier because it'll be used forever, then probably there's no amount of training that's good enough.

Re: Transformer architecture optimized for Apple Silicon

#166

Post by calvinmorrison »

ThorsBane wrote: post #137
endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

This will be the AI killer app for whoever pulls it off first. GPT-4 or higher performance on a device as nice as an iPhone...will create so many new capabilities for smart phones. It's going to be an amazing time for tech in the near future and beyond.

why does this matter when we have cloud computing and high speed mobile internet? why does it need to be local

Re: Transformer architecture optimized for Apple Silicon

#167

Post by thallium205 »

highwaylights wrote: post #72
brokencode wrote: post #68

Earlier quoted context omitted.

Nvidia is also no David to Apple’s Goliath. They are both Goliaths who will most likely be battling to build the best hardware for LLMs in the near future.

It’s an exciting time. I think people are very prematurely counting AMD and even Intel out of this race considering the pace of change.

Intel is getting way too comfy soaking in government subsidies. It’s definitely out on the innovation front.

Re: Transformer architecture optimized for Apple Silicon

#168

Post by ThorsBane »

calvinmorrison wrote: post #166
ThorsBane wrote: post #137

Earlier quoted context omitted.

This will be the AI killer app for whoever pulls it off first. GPT-4 or higher performance on a device as nice as an iPhone...will create so many new capabilities for smart phones. It's going to be an amazing time for tech in the near future and beyond.

why does this matter when we have cloud computing and high speed mobile internet? why does it need to be local

Because it's more seamless and then integrates into the Apple hardware ecosystem due to AirDrop and other integrations.

Re: Transformer architecture optimized for Apple Silicon

#169

Post by todd3834 »

endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

I’m a huge Apple fanboy, I even moved my family up to the Bay Area to work for them for a time. Hopefully you’re right but if Siri is any indicator for some reason Apple can’t seem to crack AI at the same level as OpenAI has proven to be able to.

Re: Transformer architecture optimized for Apple Silicon

#170

Post by lern_too_spel »

option wrote: post #95
endisneigh wrote: post #7

i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally. openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that opena…

I’ll give it 2 years max to reach ChatGPT like capabilities running locally on latest iPhones and Macs. Lets hope Apple doesn’t screw this up.

That's skating where the puck is. By that point, we'll be using trillion parameter models in the cloud that make ChatGPT look like a toy.
Post reply on HN