Earlier quoted context omitted.
It's not an LLM. The first L stands for large and this isn't.
What's the threshold to be called Large?
As soon as it exceeds medium.
Apple’s new Transformer-powered predictive text model
181–190 of 268 posts
Re: Apple’s new Transformer-powered predictive text model
#182Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?
It's not LLM's writing for us, it's just autocomplete. If the suggestion doesn't match what you were already planning on saying, you just ignore it. The human desire to be original and authentic is always going to be stronger. (It's much less effort to ignore it and keep typing your original thought, than it is to think about it, compare with what you were going to say, decide its version is better, and then accept i…
2 problems with suggestions:
1) there is a sort order to them. And we don't know how the 'recommendations' work. What is a recommendation? Why is Google recommending something over something else?
2) it bogs down creativity. You end up not thinking and accepting the suggestion as 'good' enough.
Re: Apple’s new Transformer-powered predictive text model
#183Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?
I don't think people write much original thought on phones anyway.
Yeah. I can’t wait to see what your parents said about desktop computers.
Re: Apple’s new Transformer-powered predictive text model
#184Earlier quoted context omitted.
It's not LLM's writing for us, it's just autocomplete. If the suggestion doesn't match what you were already planning on saying, you just ignore it. The human desire to be original and authentic is always going to be stronger. (It's much less effort to ignore it and keep typing your original thought, than it is to think about it, compare with what you were going to say, decide its version is better, and then accept i…
2 problems with suggestions: 1) there is a sort order to them. And we don't know how the 'recommendations' work. What is a recommendation? Why is Google recommending something over something else? 2) it bogs down creativity. You end up not thinking and accepting the suggestion as 'good' enough.
Most writing does not need creativity. It is wrote communication. If you want to put some creativity into something or write something where you want to write with some personality, then turn off the predictions or use it just for spell check.
Re: Apple’s new Transformer-powered predictive text model
#185Earlier quoted context omitted.
I don't think such penalties were applied to GPT-2 or even GPT-3, yet they weren't repetitive like that.
Yes, they are applied. Here's OpenAI doc which describes how to set various sampling parameters for GPT-3: https://platform.openai.com/docs/api-reference/completions/c... See presence_penalty and frequency_penalty. Sampling techniques is one of important arts of LLMs, you'll can find a lot of papers on them. In general, smaller are more prone to repetition, but you can get caught in it even with larger models.
To clarify: I meant that in general they are commonly applied, but in this case they weren't as the author confirmed. The repetition, of course, doesn't happen all the time.
Re: Apple’s new Transformer-powered predictive text model
#186Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?
It’s an important question, but one we’re not addressing in so many areas. We’re 8 billion in the world and more culturally homogeneous than ever.
I think and hope the pendulum will swing back eventually, but my guess is that it’s still got some way to go before that.
Re: Apple’s new Transformer-powered predictive text model
#187Earlier quoted context omitted.
It's not LLM's writing for us, it's just autocomplete. If the suggestion doesn't match what you were already planning on saying, you just ignore it. The human desire to be original and authentic is always going to be stronger. (It's much less effort to ignore it and keep typing your original thought, than it is to think about it, compare with what you were going to say, decide its version is better, and then accept i…
2 problems with suggestions: 1) there is a sort order to them. And we don't know how the 'recommendations' work. What is a recommendation? Why is Google recommending something over something else? 2) it bogs down creativity. You end up not thinking and accepting the suggestion as 'good' enough.
My job requires me to correspond with dozens of people over email, Teams, and Slack every day. We're all trying to get work done and need our communications to be as succinct as possible. Sure, occasionally I might dress it up to add some humor, but that's ~1% of cases. An AI, with access to my entire corpus of work-related communication, could likely very easily predict most of my communications, since they fall into a small set of categories.
"When can I expect to get $workproduct?"
"Here's when I can commit to getting you $workproduct"
"What's the estimated date for $milestone?"
"Here's the project plan for $initiative"
"I can't make this meeting, can you be sure to record it?"
I welcome any tool that can predict what I want to type and does it for me. I'm not sure if it's my imagination or not but Outlook and Teams seem to have gotten better in this regard. I'll take more of that.
Re: Apple’s new Transformer-powered predictive text model
#188Earlier quoted context omitted.
In typical Apple fanboy fashion they are oblivious to what Google has been doing for over two years now.
Apple put neural in their silicon a full four years before Google did: https://www.apple.com/newsroom/2017/09/the-future-is-here-ip... https://blog.research.google/2021/11/improved-on-device-ml-o... Apple has just been more methodical to the rest of the ecosystem - essentially waiting to understand use-cases before fully embracing it across their ecosystem from Apple Silicon in Mac with neural and `device=mps`, CoreM…
> Apple put neural in their silicon a full four years before Google did:
You're confusing when Google started designing its own mobile SOC with when Android devices (including Google's) first started using neural network accelerators, which happened months earlier on Android than on iPhones.
https://www.qualcomm.com/news/onq/2017/01/tensorflow-machine...
As for GP's claim, Android has indeed had what the iPhone is now getting for two years: https://blog.research.google/2021/10/grammar-correction-as-y...
Re: Apple’s new Transformer-powered predictive text model
#189I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…
For me stock autocorrect was always pretty much unusable, but I have no real complaints about the app I use. I'm bilingual and I type in both languages a lot. I remember when I got my first smartphone I bought SwiftKey app (later bought by Microsoft) that made language switching very easy by just swiping the space. Later the app got so good I just left it on default setting and it would still recommend the right word…
I also use SwiftKey and can highly recommend. I use Polish and English, sometimes within the same message and native keyboard was an absolute shitshow for me. It would literally never correct properly, sometimes make up non existing words etc. My only pain point with SwiftKey is that since last update it crashes often, I hope they might actually fix it someday. GBoard on iOS was forgotten long time ago so that's a no go...
Re: Apple’s new Transformer-powered predictive text model
#190I've been arguing this is the way AI should be deployed. Rather than trying to sell ai as an end to end solution, just let it do the small part that it can reliably do. It's cost effective for the host, and valuable for the user. win win engineering!
It makes a lot more economical sense to cloud-host big models like GPT-4 and optimize them for the hardware they run on. But for small models, sure, let the user run them locally, eliminating network latency.
I wouldn’t want to send everything I type to apple’s auto suggest server. For me it’s quite important that this is not apple’s approach to privacy.