Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new Transformer-powered predictive text model

jackcook.com

Post reply on HN
Print view View original

261–268 of 268 posts

Re: Apple’s new Transformer-powered predictive text model

#261

Post by catiopatio »

idk1 wrote: post #259

Does anyone know if the regular spell checker has been improved. I'm terrible at spelling and I find Apple's to be the worst, for example in the link I've shown what happens when I type "nessisary" [1] (lastest iOS 16) which is a word I often get wrong and have to Google. I feel as though this is a very clear and obvious typeo that should be picked up. If anyone is wondering what the word I'm trying to spell is, it's…

I can only hope so. Apple’s spell checker is terrible.

My word is “bureaucrat”. No matter how many times I look it up, I can never remember how to spell it.

I always try some variation of “beurocrat”, find that spell-check is of no use whatsoever, paste it into Google, and get the proper spelling instantly.

Re: Apple’s new Transformer-powered predictive text model

#262

Post by cztomsik »

toxik wrote: post #124
dchest wrote: post #92

Earlier quoted context omitted.

Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks applied, such as repetition/frequency penalty.

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

It is very common for small models, and sometimes even 7B model gets stuck.

There was an article/paper showing that GPTs (whole family) quickly get confident in looping if there is anything loop-like in the window. So basically, as soon as it loops once, it will never get out of it.

Note that sometimes loops are desired, like with docstring in the code, always starting before the function definition, and other structural things.

Re: Apple’s new Transformer-powered predictive text model

#263

Post by cztomsik »

avianlyric wrote: post #206
toxik wrote: post #124

Earlier quoted context omitted.

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops. That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to sel…

Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for.

My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough.

Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.

Re: Apple’s new Transformer-powered predictive text model

#264

Post by kevincox »

fh9302 wrote: post #154
bloppe wrote: post #152

Earlier quoted context omitted.

GPT-2 saw "today," and thought "this must be news copy" and generated more news copy. Given a few more words, it could have narrowed down the context. The Apple suggestions aren't even grammatically correct, seemingly no different from the shallow statistical completion we've already had for years, so it's weird that they branded it in lofty AI terms

Autocorrect doesn't suggest whole sentences so it is irrelevant if the remaining sentence is gibberish or not.

Or it could be the contrary. The new feature doesn't suggest whole sentences because the model they are using produces gibberish. It is quite possible that if the model was better then the would allow it to suggest longer phrases.

Re: Apple’s new Transformer-powered predictive text model

#265

Post by foobiekr »

Toutouxc wrote: post #21

Meanwhile my language and dozens of others with hundreds of millions of speakers still don’t get the predictions at all, don’t support multi-language typing and don’t support swipe typing. Typing on a $1000 iPhone in one of these languages is still a 2007 experience.

Czech has around 10M speakers world wide. It sucks that they don't have solid predictions, but there are languages with massively more linguistic footprint that will end up prioritized.

Without putting too fine a point on it, I can tell you from direct experience that while it might seem that purely statistical methods are sufficient for predictive text, one of the problems is that you're building those models (whether using neural nets or pure stat models) from a body of text that is fraught with peril and the details matter.

There's really basic stuff like profanity and other offensive constructs (see all the jokes about google suggestion auto-complete where sexist, racist, and otherwise offensive suggestions get tons of attention), then there's the problem of source languages often having a clear shift in acceptable completions for entirely different reasons.

The euphemism treadmill in US English are an obvious and continuing expense, but consider whether USSR-era Czech texts, which are substantial, would be a safe bet in training a completion system.

Re: Apple’s new Transformer-powered predictive text model

#266

Post by kridsdale3 »

bloppe wrote: post #152
Zinu wrote: post #104

The example at the end made me wonder if Apple's model is actually better than GPT2 for text prediction. It generated garbage, but all that garbage made somewhat sense in the context of only the word "Today". Whereas GPT2 hallucinated random stuff about the US government. A text prediction model should predict what the user wanted to type, so if you evaluate the models based on that, GPT2 actually performed horribly,…

GPT-2 saw "today," and thought "this must be news copy" and generated more news copy. Given a few more words, it could have narrowed down the context. The Apple suggestions aren't even grammatically correct, seemingly no different from the shallow statistical completion we've already had for years, so it's weird that they branded it in lofty AI terms

I suspect someone (Craig even) was under some pressure from The Board to have >0 references to generative-AI in their presentation this year since every single company (even non-software) is now expected by Wall St to "be doing some AI". Even though Apple is at the top of the heap with ML in photography and many other domains, without some kind of LLM the tech news narrative will be "Apple is years behind".

Re: Apple’s new Transformer-powered predictive text model

#267

Post by xp84 »

tornato7 wrote: post #207
xp84 wrote: post #197

I want to know if this will be used to improve all the places where Apple devices attempt to interpret what you might mean to type, including the swipe keyboard. I've been suffering for years with their terrible, unusable swipe typing. You can't even get it to swipe "I love you" because it always prioritizes "your" over you, regardless of the context. I've even experimented extremely slowly and taken screen recording…

I love you I love you I love you I love you Seems to work fine. Though I typically use Gboard because the swipe typing is much better. I find the most restrictive thing about Apple's autocorrect and speech to text to be the limited vocabulary. Once you start using any industry terms it completely fails.

Especially to whomever downvoted me: there are at least 64 of us with this same bug.[1] — and that’s just the ones who bothered to waste time on Apple’s official discussion board.

I reset my keyboard dictionary again on iOS 17, still exact same reproducible bug. It’s been at least 2 years.

[1]. https://discussions.apple.com/thread/253315276

Re: Apple’s new Transformer-powered predictive text model

#268

Post by IndySun »

IndySun wrote: post #77

Apple's current autocorrect is skewed towards verbal, not typo, errors. Apple wants your voice. Implementing this deliberately worse change also gave Apple a path to 'improve autocorrect'. Comments are correct, T9 was based on the keyboard and common mistyped keys, simple, and effective.

Don't take my word for it, as you all blindly have not.

Take the word 'lie', often mistyped as 'loe'. This is because the o is next to the i. After accidentally typing 'loe', look at what iOS offers to replace it. low, lot, Joe. Soundalikes. This is because iOS autocorrect is skewered towards spoken vocabulary and not commonly mistyped words in english on a typical 1874 Sholes & Glidden qwerty layout. T9 worked because of this distinction. And iOS does not work as consistently because of this distinction.

My 'feeling' is that Apple were very much aware of this from the iOS outset, and not wanting to use T9, offered this poorer auto-correct, and are now riding over the hill shouting 'we've solved it'.

This may not be so for other languages.

Post reply on HN