131–140 of 268 posts
In typical Apple fashion, when everyone's going bigger and stronger, they're going in a different direction... With optimising to the smallest model that can run all day without draining your battery. I love that they're almost never first to market but they find a way to distil value than others don't. It's the almond milk of technology.
Earlier quoted context omitted.
Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks applied, such as repetition/frequency penalty.
I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.
Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2
I've been arguing this is the way AI should be deployed. Rather than trying to sell ai as an end to end solution, just let it do the small part that it can reliably do. It's cost effective for the host, and valuable for the user. win win engineering!
That said, as a user, it is nice to see LLMs used in a small, discrete, undeniably useful way like this. No flashy promises, nothing new for me to learn- it’s just autocorrect, but better.
Meanwhile my language and dozens of others with hundreds of millions of speakers still don’t get the predictions at all, don’t support multi-language typing and don’t support swipe typing. Typing on a $1000 iPhone in one of these languages is still a 2007 experience.
Even worse, on the Apple Watch you cannot input text at all in one of these languages, as you cannot (!) turn autocorrect off on the watch. They managed to fit a full QWERTY keyboard on the screen yet made it completely useless.
Earlier quoted context omitted.
Word (or multi-word) prediction is a great starting place for an autocorrect model. If the keyboard I'm using knows the probability of all the possible next tokens I could type, then you can start making the tap targets for those keys bigger. And since the model is super cheap to run, you can replay the last n keyboard taps to simulate a different correction being made. That's at least how I'd make autocorrect better…
> then you can start making the tap targets for those keys bigger That’s how autocorrect has worked since day one, it was just using a simpler heuristic-based prediction rather than a machine learning model.
The solution badly needed isn't a complicated black box model when we know a simple one is good enough.
Earlier quoted context omitted.
Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks applied, such as repetition/frequency penalty.
Such penalties would only exist for fine-tuned models. Base models have only the temperature setting. As the example at the end shows, even GPT-2 seems "smarter" than the Apple model, probably because of the number of parameters.
For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp...
Here's the one from transformers: https://github.com/huggingface/transformers/blob/0a55d9f7376...
To summarize how they work: you keep some number of previously generated tokens, and once you get logits that you want to sample a new token from, you find the logits for existing tokens and multiply them by a penalty, thus lowering the probability of the corresponding tokens.
Is it possible for it to improve based on experience/iMessage history or is it locked in? I’d love to opt-in to training it on my previous convos…
It already does and I hate it. If a person you’re speaking with misspells a word, it’ll happy suggest^w forcibly autocorrect a similar word to that misspelling if it was recent enough.
I don't want better predictive text. I need better autocorrect. Something happened about 6 years ago where the quality of the autocorrect fell off the roof, and it's been absolutely terrible since then. I spend too much of my time fighting with either mispelling, flipping to the wrong word even though I spelled the word properly, etc. It has made typing on my iPhone an unpleasant experience and I need it to change. T…
I use SwiftKey and it's much better than the Apple one. But it does require giving Microsoft access to most of what you write.
Is this what progress looks like?
Earlier quoted context omitted.
Out of curiosity, what accent do you have? In my experience that can have a significant impact on how good speed to text works for you.
Fairly neutral British. Born and raised in Guernsey so have a complete absence of regional accent without the very forced enunciation that comes from elocution lessons / Queens English. Admittedly I probably represent the best-case subject.
I don't think you could do any better unless you were from Cupertino. With an Irish accent, my experience has been... okay with Apples text to speech, it's certainly better than others, but I still have to make a conscious effort to enunciate quite differently to my normal speech.
OpenAI's whisper has really impressed me though, and transcribes almost everything perfectly, even if I throw in phrases or words from other languages part way through the conversation.