Apple’s new Transformer-powered predictive text model • Hacker News Board

Re: Apple’s new Transformer-powered predictive text model

#161

Post by macNchz » Sun, Sep 17, 2023, 1:58 PM UTC

astrange wrote: post #150

tuukkah wrote: post #123

Earlier quoted context omitted.

Turn off autocorrect: https://support.apple.com/en-us/HT207525

Why? So you can use text speak? Bit silly.

I’ve had autocorrect disabled on my iPhone for years—for whatever reason it really bothers me when something I just typed magically changes into something I didn’t want to write. I like that what I type is exactly what appears, not the computer’s guess at my intent. It’s one of the first things I disable when setting up a new Mac as well.

It took a few months to get used to it, but I developed typo-correction muscle memory pretty quickly (RIP 3D Touch, it was awesome for text editing). Plus I think that the copyedit re-read that I give to things I write on my phone is helpful above and beyond just catching typos. So be it if I make the occasional typo!

Re: Apple’s new Transformer-powered predictive text model

#162

Post by kayodelycaon » Sun, Sep 17, 2023, 2:00 PM UTC

tuukkah wrote: post #155

astrange wrote: post #150

Earlier quoted context omitted.

Why? So you can use text speak? Bit silly.

In general so that what you write is not mutilated by software that doesn't know the words you use. If there are typos, it's not difficult for the receiver to "autocorrect" while reading, but when autocorrect miscorrects, it's not easy for the writer to notice how the message changed nor for the reader to guess what was there before the "fixes".

That assumes your typing and spelling accuracy is high and the other person reading has significant proficiency in your language.

My typing and spelling accuracy is horrendous to the point I use autocorrect on my laptop. For me, it’s an important accessibility feature. I also use dictation for single words.

Re: Apple’s new Transformer-powered predictive text model

#163

Post by dbbk » Sun, Sep 17, 2023, 2:09 PM UTC

M4v3R wrote: post #112

layer8 wrote: post #111

Earlier quoted context omitted.

I don’t understand why people still put up with Apple’s autocorrect instead of just turning it off. It’s so wrong so often as to be utterly useless.

Because it depends on the person. For me autocorrect is 99% spot on and so it’s a valid trade off for me to deal with correcting the remaining 1%. My wife on the other hand cannot live with it so she turned it off. And every time I type something on her phone I immediately feel slower because I know each of my key presses have to be much more deliberate.

In recent years mine has gone completely crazy and started autocorrecting to random words that don't exist. I think somehow over time it can get corrupted.

Re: Apple’s new Transformer-powered predictive text model

#164

Post by flangola7 » Sun, Sep 17, 2023, 2:12 PM UTC

astrange wrote: post #151

apozem wrote: post #133

Earlier quoted context omitted.

I mean, I get why all these startups are trying to sell AI as a panacea. It’s an exciting technology and someone has to figure out its limits. That said, as a user, it is nice to see LLMs used in a small, discrete, undeniably useful way like this. No flashy promises, nothing new for me to learn- it’s just autocorrect, but better.

It's not an LLM. The first L stands for large and this isn't.

What's the threshold to be called Large?

Re: Apple’s new Transformer-powered predictive text model

#165

Post by basisword » Sun, Sep 17, 2023, 2:19 PM UTC

tuukkah wrote: post #123

basisword wrote: post #113

Earlier quoted context omitted.

You use text speak on an iPhone? How? I find it impossible due to autocorrect. Either it will correct with the full word or it will correct with a wrong word making it more efficient to just type in full.

Turn off autocorrect: https://support.apple.com/en-us/HT207525

My problem with this (I’ve tried it) is that the iOS keyboard still seems to try to guess which key you meant to type. So you hit “o” and it thinks you probably meant “p” and inserts that. Last I checked this still occurred with autocorrect off and made it even worse than using autocorrect.

Re: Apple’s new Transformer-powered predictive text model

#166

Post by flangola7 » Sun, Sep 17, 2023, 2:20 PM UTC

kkielhofner wrote: post #157

jaimex2 wrote: post #130

Earlier quoted context omitted.

In typical Apple fanboy fashion they are oblivious to what Google has been doing for over two years now.

Apple put neural in their silicon a full four years before Google did: https://www.apple.com/newsroom/2017/09/the-future-is-here-ip... https://blog.research.google/2021/11/improved-on-device-ml-o... Apple has just been more methodical to the rest of the ecosystem - essentially waiting to understand use-cases before fully embracing it across their ecosystem from Apple Silicon in Mac with neural and `device=mps`, CoreM…

How does Snapchat use it?

Re: Apple’s new Transformer-powered predictive text model

#167

Post by QuadrupleA » Sun, Sep 17, 2023, 2:20 PM UTC

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

Re: Apple’s new Transformer-powered predictive text model

#168

Post by tempaccount420 » Sun, Sep 17, 2023, 2:22 PM UTC

QuadrupleA wrote: post #167

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

I don't think people write much original thought on phones anyway.

Re: Apple’s new Transformer-powered predictive text model

#169

Post by dartos » Sun, Sep 17, 2023, 2:24 PM UTC

QuadrupleA wrote: post #167

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

Most sort messages people send on Apple devices aren’t meant to be original, just to quickly convey some information.

Re: Apple’s new Transformer-powered predictive text model

#170

Post by smpanaro » Sun, Sep 17, 2023, 2:24 PM UTC

codeflo wrote: post #5

And interesting test. I didn’t see a mention of the temperature setting used. Temperature controls the probability to pick a token that isn’t the top prediction, which leads to more creative/less robotic results. For actual input prediction, you probably want the temperature to be zero. But even a model as good as GPT-3 becomes very boring and repetitive with those settings.

There is an input for temperature in the CPU model. If you can find and hook the call to predict you can probably see what is being passed.

Interestingly the Neural Engine version of the model does not take a temperature input, but it does output the raw embeddings in addition to the logits.