What's the data corpus? I'm very surprised that the words "Be," "Of," and "And" are among the 25 most common first words of sentences.
Show HN: Predictive text using only 13kb of JavaScript. no LLM
21–30 of 42 posts
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#22Nice work!
> If it can do this in 13kb, it makes me wonder what it could do with more bytes.
Maybe I misunderstand, but is this not just the first baby steps of an LLM written in JS? "what it could do with more bytes" is surely "GPT2 in javascript"?
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#23What does that mean no LLM? Markov chains have been a thing for a long time and predictive text was used before LLMs.
[deleted]
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#24Nice work! > If it can do this in 13kb, it makes me wonder what it could do with more bytes. Maybe I misunderstand, but is this not just the first baby steps of an LLM written in JS? "what it could do with more bytes" is surely "GPT2 in javascript"?
Author had LLM help them make a tree of words, and the algo choose which node we're at and offers children as completions. It's clever and cute but, not even close to an LLM.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#25Seeing that you use Obsidian: Have you thought about turning that into an Obsidian plugin that offers the predictions as editor suggestions?
This exists on a word level.
Completr is a plugin that provides auto-completion functionality for obsidian.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#26Seeing that you use Obsidian: Have you thought about turning that into an Obsidian plugin that offers the predictions as editor suggestions?
This exists on a word level. Completr is a plugin that provides auto-completion functionality for obsidian. https://github.com/tth05/obsidian-completr
That does change completions from a list of words, but not completions based on predictions of the next word.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#27What's the data corpus? I'm very surprised that the words "Be," "Of," and "And" are among the 25 most common first words of sentences.
I think the LLM that helped coauthor the tree hallucinated these suggestions. Probably better to actually train it on some text.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#28Earlier quoted context omitted.
Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.
That's a markov chain of the 1st order. This is higher order Markov chain
I think Markov chains have transition probabilities, which this model is lacking. But it's the same idea, just with uniform transition probabilitis.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#29"What if we go to a place where there is no way" Barely got 1 or 2 suggestions while typing that. Am I holding it wrong?
I tried for "once in a while we all get sad" and it got exactly zero.
Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM
#30Pretty cool, a js implementation of n-grams! What is amazing to me is this: imagine that English only had 10,000 words. For each of those 10,000 words there’s 100 valid subsequent words. So there’s 1 million valid bigrams. Now if you want trigrams that takes you to 100 million, and for 4-grams it’ll be 10 billion. Just for that, you’d need 14 bytes per word and gigabytes of storage. LLMs typically have context window…
I mean ... technically Gemini 1.5 has 10 million context which just pushes the insanity further lol