Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new Transformer-powered predictive text model

jackcook.com

Post reply on HN
Print view View original

191–200 of 268 posts

Re: Apple’s new Transformer-powered predictive text model

#191

Post by QuadrupleA »

brookst wrote: post #177
QuadrupleA wrote: post #167

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

Most human communication is bland, and people who make a point of being unpredictable and shocking are usually pretty annoying. Think of it like spellcheck : the vast majority of the time it produces desired results, but if you really want to type bjPvc9fQ, you certainly can.

Well it's not so much about deliberately / affectedly being original and weird (which is annoying), but just leaving some space for natural idiosyncratic ways of writing.

Re: Apple’s new Transformer-powered predictive text model

#192

Post by QuadrupleA »

brookst wrote: post #180
QuadrupleA wrote: post #175

Earlier quoted context omitted.

That's just temperature, which evens out the random probability a little of the N most probable next words. It's still vastly favoring the N most common ones based on the training corpus, and will have a hard time producing uncommon ones. E.g. try asking an LLM to name a real, non-famous person. The internet and it's training corpus is full of regular people, but you won't have much luck - they're statistically too u…

> That's just temperature, which evens out the random probability a little of the N most probable next words. Source? A MS exec said that creative and precise are GPT4 but balanced is a not (or not 100%): https://twitter.com/MParakhin/status/1693579775590224097

I was assuming those Bing Chat settings are temperature-based, those types of "creative/precise" controls usually are - but perhaps there's more to it.

Re: Apple’s new Transformer-powered predictive text model

#193

Post by dkga »

jackcook wrote: post #148

Wow, it was quite the surprise to wake up to seeing this post near the top of HN! I wrote the post, happy to answer questions if anyone is wondering about any details.

Well, that was pretty cool. Plus, I got to know about the way processes communicate via xpc and that opened a whole new rabbit whole!

Re: Apple’s new Transformer-powered predictive text model

#194

Post by quest88 »

brookst wrote: post #177
QuadrupleA wrote: post #167

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

Most human communication is bland, and people who make a point of being unpredictable and shocking are usually pretty annoying. Think of it like spellcheck : the vast majority of the time it produces desired results, but if you really want to type bjPvc9fQ, you certainly can.

Is this an apt analogy? Don't LLMs train off these bland humans you mention? Wouldn't LLMs then also be bland?

Re: Apple’s new Transformer-powered predictive text model

#195

Post by barathr »

We might ask ourselves at some point: if the point of simple text prediction is to speed up text entry, and not generate large amounts of text from scratch, then that's a sign the input interfaces are the bottleneck. If we had a way to get text from our brains to the computer faster, we wouldn't need this kind of prediction.

Re: Apple’s new Transformer-powered predictive text model

#196

Post by jsolson »

QuadrupleA wrote: post #167

Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?

I find they're useful as a tool for achieving a particular structure or tone that isn't "my voice," including populating boilerplate from bullet points. Sometimes I'll go back and revisit everything they've produced, but they let me put something tolerable in place early. This lets me focus on the meaty parts of the text sooner than I'd otherwise find myself capable of.

I personally don't find them useful for quick / short / informal communication like email, or at least not yet.

Re: Apple’s new Transformer-powered predictive text model

#197

Post by xp84 »

I want to know if this will be used to improve all the places where Apple devices attempt to interpret what you might mean to type, including the swipe keyboard. I've been suffering for years with their terrible, unusable swipe typing. You can't even get it to swipe "I love you" because it always prioritizes "your" over you, regardless of the context. I've even experimented extremely slowly and taken screen recordings to ensure the swipes are accurate. There are a couple of other completely astonishing common words that it 100% of the time gets wrong. I guess GPT3 would probably tax the battery, but on the other hand, I'd like to have the freedom to try it, because it would let me finish tasks much easier if text entry wasn't like fighting with an insane overconfident toddler. Honestly, I don't know what happened. Text entry with iPhones 10 years ago was far less infuriating.

Re: Apple’s new Transformer-powered predictive text model

#198

Post by jsolson »

quest88 wrote: post #194
brookst wrote: post #177

Earlier quoted context omitted.

Most human communication is bland, and people who make a point of being unpredictable and shocking are usually pretty annoying. Think of it like spellcheck : the vast majority of the time it produces desired results, but if you really want to type bjPvc9fQ, you certainly can.

Is this an apt analogy? Don't LLMs train off these bland humans you mention? Wouldn't LLMs then also be bland?

Yes, but sometimes that's what's called for. I think I may have found my answer to where LLMs might be useful for short communications: softening something that might be interpreted as "curt" or even "rude" while not really changing the message.

Re: Apple’s new Transformer-powered predictive text model

#199

Post by pmcf »

xp84 wrote: post #197

I want to know if this will be used to improve all the places where Apple devices attempt to interpret what you might mean to type, including the swipe keyboard. I've been suffering for years with their terrible, unusable swipe typing. You can't even get it to swipe "I love you" because it always prioritizes "your" over you, regardless of the context. I've even experimented extremely slowly and taken screen recording…

You have to be ducking kidding. Works great for me.

Re: Apple’s new Transformer-powered predictive text model

#200

Post by xp84 »

QuadrupleA wrote: post #175
elicash wrote: post #173

Earlier quoted context omitted.

"Bing Chat's" implementation already allows you to select more creative generation of text. It's just a radio button option. There are also different technical solutions for the LLM to select which word to generate that allow either for more interesting, or more predictable, words. This isn't to say a human element doesn't have a ton to offer! Just to say that we aren't necessarily engineering a bland world of pablum…

That's just temperature, which evens out the random probability a little of the N most probable next words. It's still vastly favoring the N most common ones based on the training corpus, and will have a hard time producing uncommon ones. E.g. try asking an LLM to name a real, non-famous person. The internet and it's training corpus is full of regular people, but you won't have much luck - they're statistically too u…

I would be skeptical of the results of that experiment, just because I assume the minders of the big LLMs have attempted to make it deeply uncomfortable with discussing anything that might be "personal." For a fun time, ask ChatGPT who was executed (as in capital punishment) in the US in a certain long-past year. It responds with a bunch of rubbish about privacy and about how an LLM can't be completely sure about stuff so it wouldn't be ok to speculate for fear of tarnishing someone's reputation, as though the person executed 15 years ago is going to sue OpenAI for sharing their name and what crime they were publicly convicted of and killed for.
Post reply on HN