Apple’s new Transformer-powered predictive text model

Post by artursapek » Sun, Sep 17, 2023, 11:34 AM UTC

jonplackett wrote: post #64

blindriver wrote: post #17

I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…

Totally agree. And I’d like to add that the editing experience is also dire and getting worse. Just trying to put the cursor where I want it to fix an error earlier in the sentence in ios17 is a massive pain in the ass. You tap somewhere and it continually selects entire words - I just want to put the cursor there dammit!

Post by cubefox » Sun, Sep 17, 2023, 11:42 AM UTC

dchest wrote: post #137

cubefox wrote: post #118

Earlier quoted context omitted.

Such penalties would only exist for fine-tuned models. Base models have only the temperature setting. As the example at the end shows, even GPT-2 seems "smarter" than the Apple model, probably because of the number of parameters.

There's no such thing as "base models have only the temperature setting". Models do not have sampling settings (temperature, repetition penalty, etc), the sampling code does, which obviously you can use on any model. For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp... Here's the one from transformers: https://github.com/hugging…

Post by jackcook » Sun, Sep 17, 2023, 12:02 PM UTC

codeflo wrote: post #5

And interesting test. I didn’t see a mention of the temperature setting used. Temperature controls the probability to pick a token that isn’t the top prediction, which leads to more creative/less robotic results. For actual input prediction, you probably want the temperature to be zero. But even a model as good as GPT-3 becomes very boring and repetitive with those settings.

Post by jackcook » Sun, Sep 17, 2023, 12:06 PM UTC

cubefox wrote: post #143

dchest wrote: post #137

Earlier quoted context omitted.

There's no such thing as "base models have only the temperature setting". Models do not have sampling settings (temperature, repetition penalty, etc), the sampling code does, which obviously you can use on any model. For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp... Here's the one from transformers: https://github.com/hugging…

I don't think such penalties were applied to GPT-2 or even GPT-3, yet they weren't repetitive like that.

Post by petesergeant » Sun, Sep 17, 2023, 12:18 PM UTC

layer8 wrote: post #108

petesergeant wrote: post #97

Earlier quoted context omitted.

Yes. Ever since I went to the Maldives it wants to capitalize "male" and I have to work quite hard to get it to not correct "Yes" to "Yea", and "Thailand" to "Thailnd". Hate it.

In Settings there is Reset Keyboard Dictionary to revert to the default dictionary.