Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new Transformer-powered predictive text model

jackcook.com

Post reply on HN
Print view View original

251–260 of 268 posts

Re: Apple’s new Transformer-powered predictive text model

#251

Post by BoorishBears »

xp84 wrote: post #201
cubefox wrote: post #116

Earlier quoted context omitted.

The example at the end sounds just like the predictions you get from normal phone keyboards in the last couple of years, which presumably don't use a modern GPT-style language model. A bit disappointing.

Seriously disappointing. I was expecting that it would not produce total gibberish. It acts like it's a Markov chain, and only considers the last 1-2 words. Identical to the currently-shipping thing that we've had for the past however-many years.

People trying to draw this comparison proves making good products is harder than it seems...

The default goal everyone is assuming is spitting out the longest correct sequence possible.

But in reality the mental cost of a wildly wrong prediction is much higher than the mental cost of a slightly wrong one, so what you'd train the model for is sequences of a few words at most being with higher confidence.

Most people can/will tune out slightly wrong words especially as they get a feel for what autocorrect is good and bad at.

If you unleash the full range of tokens GPT 2 can normally output, you'll constantly be blasting out words they didn't expect.

—

The fact your long sequence prediction got better doesn't matter because the UI is autocomplete not "auto-write": they're still expecting to drive, and a smart but noisy copilot is worse than a dumb but lazy one in that case.

I wouldn't be surprised if they trained the model to an effective context window of just a few hundred tokens with that in mind

Re: Apple’s new Transformer-powered predictive text model

#252

Post by [deleted] »

kkielhofner wrote: post #248
smoldesu wrote: post #246

Earlier quoted context omitted.

It's important to ask, because your sources are not referring to the same thing. The first is a reference to training techniques that have nothing to do with CoreML or Apple hardware. The second thing you've linked is a transformer model from 2022 that was ported to CoreML (alongside Pytorch, ONNX and 10+ other execution providers). It's extremely unclear how any of these sources corroborate your claim, particularly…

Is it really so hard to believe that the technical details of Snap's implementation just may not have been made publicly available at the time they deployed it? It's not as though they're known for running a breaking engineering blog like some smaller companies... Speaking of which: In my top comment I said "Snapchat, as one example". Feel free to search around to find others that were more transparent on implementat…

[deleted]

Re: Apple’s new Transformer-powered predictive text model

#253

Post by jychang »

BoorishBears wrote: post #251
xp84 wrote: post #201

Earlier quoted context omitted.

Seriously disappointing. I was expecting that it would not produce total gibberish. It acts like it's a Markov chain, and only considers the last 1-2 words. Identical to the currently-shipping thing that we've had for the past however-many years.

People trying to draw this comparison proves making good products is harder than it seems... The default goal everyone is assuming is spitting out the longest correct sequence possible. But in reality the mental cost of a wildly wrong prediction is much higher than the mental cost of a slightly wrong one, so what you'd train the model for is sequences of a few words at most being with higher confidence. Most people c…

This comment is the summary of the difference between human driven design and technology driven design.

Too many people are focused on technology for technology’s sake.

Re: Apple’s new Transformer-powered predictive text model

#255

Post by callalex »

r0m4n0 wrote: post #232
ezfe wrote: post #224

Earlier quoted context omitted.

My iPhone has always suggested those things...

Sometimes on obvious fields like an email input on a web page (not all though). Not in this comment box for example

That doesn’t match my experience at all. I agree with you that it should “just work” and you shouldn’t have to think about it, but do you have a contact for yourself saved, and have you set it as your “me” contact in Settings>Contacts>My Info?

Re: Apple’s new Transformer-powered predictive text model

#256

Post by lern_too_spel »

kkielhofner wrote: post #250
lern_too_spel wrote: post #243

Earlier quoted context omitted.

> A press release from Qualcomm that came a few months before Apple actually got it in people's hands This press release was for a new Tensorflow release for devices that had been shipping for many months before that. It was in people's hands long before Apple even announced anything similar. > If not that's on the Gboard team at Google for not supporting neural on iOS and Apple hardware that came far, far sooner. On…

All this thread has taught me is that Google/Apple religious zealotry is a very real thing. What's interesting about this "debate" is my original links very, very, very clearly show the reality: Google being excited about launching the Pixel 6 (in 2021) with Tensor silicon - a first for them. All anyone on this thread has done since is refuse to acknowledge how obvious and clear that is while deflecting and throwing…

What was a first for Google was having its own silicon at all. The article was about how Google designed models specifically for it with neural architecture search, something Apple still doesn't do.

What's clear to me is you're salty over Apple continuously being behind to the point that you misread articles in order to support your fanaticism and refuse to acknowledge it even after you've been shown the evidence.

Re: Apple’s new Transformer-powered predictive text model

#257

Post by lern_too_spel »

kkielhofner wrote: post #249
lern_too_spel wrote: post #245

Earlier quoted context omitted.

Your linked source says Snapchat has been using CoreML for neural network acceleration since 2022, which is not "since the early days" as you had originally claimed.

The link was provided to refute "Snap uses CPU". The repo is from 2022 but do you really think Snap publishes breaking code and papers on new, novel, and highly competitive functionality? The snap-research GH org wasn't even created until 2020. Do you really think they weren't doing anything before that because it's not on Github? This thread has been exhausting, I suspect due to the religious war that is Apple/Googl…

It's bizarre how you keep refusing to acknowledge you were wrong. You made specific claims that were specifically refuted and now you're "yeah, butting" wildly to try to somehow win an argument you started by trying to find something else that Apple might be better at.

As far as I'm concerned, the war has always (since the mid-80s) been Apple versus everybody else. Nobody else makes clearly wrong claims like the comment at the start of this thread. What yanks my chain is that Apple is against user control, and we as developers should be above falling for their marketing that they're first or best when they very clearly aren't.

Re: Apple’s new Transformer-powered predictive text model

#258

Post by r0m4n0 »

callalex wrote: post #255
r0m4n0 wrote: post #232

Earlier quoted context omitted.

Sometimes on obvious fields like an email input on a web page (not all though). Not in this comment box for example

That doesn’t match my experience at all. I agree with you that it should “just work” and you shouldn’t have to think about it, but do you have a contact for yourself saved, and have you set it as your “me” contact in Settings>Contacts>My Info?

Yes I do. But also, my wife has a different name, do I have to type out her full name every time? There is no stored learning that I have experienced whatsoever on normal typing. The little contact auto fill that you both are referring to is extremely limited, the form field literally needs to say “email” and doesn’t work out of context. As I said, did you try to type your email out in this comment response box?

Re: Apple’s new Transformer-powered predictive text model

#259

Post by idk1 »

Does anyone know if the regular spell checker has been improved. I'm terrible at spelling and I find Apple's to be the worst, for example in the link I've shown what happens when I type "nessisary" [1] (lastest iOS 16) which is a word I often get wrong and have to Google. I feel as though this is a very clear and obvious typeo that should be picked up. If anyone is wondering what the word I'm trying to spell is, it's - necessary. Google picked this up first time.

1 - https://i.imgur.com/jcfbMLn.jpeg

Re: Apple’s new Transformer-powered predictive text model

#260

Post by jingles_dev »

idk1 wrote: post #259

Does anyone know if the regular spell checker has been improved. I'm terrible at spelling and I find Apple's to be the worst, for example in the link I've shown what happens when I type "nessisary" [1] (lastest iOS 16) which is a word I often get wrong and have to Google. I feel as though this is a very clear and obvious typeo that should be picked up. If anyone is wondering what the word I'm trying to spell is, it's…

+1
Post reply on HN