Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new Transformer-powered predictive text model

jackcook.com

Post reply on HN
Print view View original

51–60 of 268 posts

Re: Apple’s new Transformer-powered predictive text model

#51

Post by nixpulvis »

blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

How about just fixing the damn spellchecker on iOS?

I can never press on underlined words that are the last word before a linebreak while typing.

It’s been broken for years. YEARS!

Re: Apple’s new Transformer-powered predictive text model

#52

Post by deergomoo »

Me1000 wrote: post #28
blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

Word (or multi-word) prediction is a great starting place for an autocorrect model. If the keyboard I'm using knows the probability of all the possible next tokens I could type, then you can start making the tap targets for those keys bigger. And since the model is super cheap to run, you can replay the last n keyboard taps to simulate a different correction being made. That's at least how I'd make autocorrect better…

> then you can start making the tap targets for those keys bigger

That’s how autocorrect has worked since day one, it was just using a simpler heuristic-based prediction rather than a machine learning model.

Re: Apple’s new Transformer-powered predictive text model

#53

Post by dkga »

Beyond fixing iPhone’s autocorrect as widely mentioned below, I wonder if in future versions Apple will end up switching to phi-1.5 or other models that are much smaller but trained on higher quality data.

Would also be cool if they trained their own copilot for Xcode, given their obviously enturmes code base in Swift and Obj-C.

Re: Apple’s new Transformer-powered predictive text model

#54

Post by deergomoo »

blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

> Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then

That was when they first changed to a machine learning-based model. I’m hoping this one is better.

Re: Apple’s new Transformer-powered predictive text model

#55

Post by tgv »

Toutouxc wrote: post #21

Meanwhile my language and dozens of others with hundreds of millions of speakers still don’t get the predictions at all, don’t support multi-language typing and don’t support swipe typing. Typing on a $1000 iPhone in one of these languages is still a 2007 experience.

Modern language processing relies on pretty large corpora, and to bootstrap it requires a part to be of high quality and annotated, although for spelling correction you don't need annotation. Or you can go the GPT-3 route and use really huge corpora. If that doesn't exist for your language, and for many smaller languages it doesn't, it won't get better than 2007.

Re: Apple’s new Transformer-powered predictive text model

#56

Post by mcintyre1994 »

ComputerGuru wrote: post #34
tpowell wrote: post #13

Is it possible for it to improve based on experience/iMessage history or is it locked in? I’d love to opt-in to training it on my previous convos…

It already does and I hate it. If a person you’re speaking with misspells a word, it’ll happy suggest^w forcibly autocorrect a similar word to that misspelling if it was recent enough.

Do they do that across conversations, eg if I misspell something while texting you does it recommend my misspelling when you’re talking to other people?

Re: Apple’s new Transformer-powered predictive text model

#57

Post by therein »

AmericanChopper wrote: post #27
blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

I regularly use 4 languages on my iPhone and my autocorrect is completely broken. I can’t be bothered switching between keyboards, so the English keyboard has just completely lost the plot…

Yup same, that's why I am sticking with Android as well. I want multi-language auto-correct for my preferred languages _on the English QWERTY keyboard_.

Re: Apple’s new Transformer-powered predictive text model

#58

Post by corbezzoli »

tgv wrote: post #55
Toutouxc wrote: post #21

Meanwhile my language and dozens of others with hundreds of millions of speakers still don’t get the predictions at all, don’t support multi-language typing and don’t support swipe typing. Typing on a $1000 iPhone in one of these languages is still a 2007 experience.

Modern language processing relies on pretty large corpora, and to bootstrap it requires a part to be of high quality and annotated, although for spelling correction you don't need annotation. Or you can go the GPT-3 route and use really huge corpora. If that doesn't exist for your language, and for many smaller languages it doesn't, it won't get better than 2007.

Indonesian is spoken by 200M people (45M of which as first language), I bet they could find some text for it. When joined with the Malaysian family, it brings the total speakers to 290M.

Re: Apple’s new Transformer-powered predictive text model

#59

Post by messe »

jamiedumont wrote: post #47
pentae wrote: post #25

Earlier quoted context omitted.

Same with voice to text translation. It's absolutely, utterly, worthless. I end up having to go and edit half the message. Apple can't even get basic things right

I broke my right (dominant) collarbone on Wednesday so I’ve entered the realm of the temporarily disabled user. So far I prefer iOS because the predictive text above the keyboard (in iOS 16 too) and in particular the speech to text feature has been so good! The only egregious error that I have encountered has been confusion around the phrase “into” when I mean “in two”. An understandable mistake.

Out of curiosity, what accent do you have? In my experience that can have a significant impact on how good speed to text works for you.

Re: Apple’s new Transformer-powered predictive text model

#60

Post by aidos »

Me1000 wrote: post #28
blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

Word (or multi-word) prediction is a great starting place for an autocorrect model. If the keyboard I'm using knows the probability of all the possible next tokens I could type, then you can start making the tap targets for those keys bigger. And since the model is super cheap to run, you can replay the last n keyboard taps to simulate a different correction being made. That's at least how I'd make autocorrect better…

Maybe it’s better and I just need to keep typing so it can figure out what I mean.

In the last couple of weeks using the beta I’ve found myself fighting it more than ever. After years of typing by tapping at the screen, I’ve switched to swiping the words in because it seems to be more reliable.

Post reply on HN