Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Predictive text using only 13kb of JavaScript. no LLM

adamgrant.info

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#31

Post by cchance »

refulgentis wrote: post #24
underscoring wrote: post #22

Nice work! > If it can do this in 13kb, it makes me wonder what it could do with more bytes. Maybe I misunderstand, but is this not just the first baby steps of an LLM written in JS? "what it could do with more bytes" is surely "GPT2 in javascript"?

Author had LLM help them make a tree of words, and the algo choose which node we're at and offers children as completions. It's clever and cute but, not even close to an LLM.

I mean it's not far off from a super low quant LLM with limited params, like a 1bit quant LLM with low params XD

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#32

Post by refulgentis »

cchance wrote: post #31
refulgentis wrote: post #24

Earlier quoted context omitted.

Author had LLM help them make a tree of words, and the algo choose which node we're at and offers children as completions. It's clever and cute but, not even close to an LLM.

I mean it's not far off from a super low quant LLM with limited params, like a 1bit quant LLM with low params XD

It's very far off, like "not even wrong" in the Pauli sense of the phrase.

There's a lot of abstractions one can have for this stuff, I think you're looking at that "text predictor" is one of them?

If you roll with that, then you're in a position where you're saying GPT-2 class LLMs were very close in 1960, because at the end of the day, it's just a dictionary lookup with a string key and a value of list completions. That confuses instead of illuminates.

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#33

Post by dartos »

Robin_Message wrote: post #28
alucab wrote: post #11

Earlier quoted context omitted.

That's a markov chain of the 1st order. This is higher order Markov chain

I think Markov chains have transition probabilities, which this model is lacking. But it's the same idea, just with uniform transition probabilitis.

It seems like it has transition probabilities.

Depending on the type of the prior word, it randomly selects the next word from a list of compatible word types.

Am I misunderstanding?

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#34

Post by throwaway4aday »

Nice work! I built something similar years ago and I did compile the probabilities based on a corpus of text (public domain books) in an attempt to produce writing in the style of various authors. The results were actually quite similar to the output of nanoGPT[0]. It was very unoptimized and everything was kept in memory. I also knew nothing about embeddings at the time and only a little about NLP techniques that would certainly have helped. Using a graph database would have probably been better than the datastructure I came up with at the time. You should look into stuff like Datalog, Tries[1], and N-Triples[2] for more inspiration.

Your idea of splitting the probabilities based on whether you're starting the sentence or finishing it is interesting but you might be able to benefit from an approach that creates a "window" of text you can use for lookup, using an LCS[3] algorithm could do that. There's probably a lot of optimization you could do based on the probabilities of different sequences, I think this was the fundamental thing I was exploring in my project.

Seeing this has inspired me further to consider working on that project again at some point.

[0] https://github.com/karpathy/nanoGPT

[1] https://en.wikipedia.org/wiki/Trie

[2] https://en.wikipedia.org/wiki/N-Triples

[3] https://en.wikipedia.org/wiki/Longest_common_subsequence

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#35

Post by Legend2440 »

cchance wrote: post #31
refulgentis wrote: post #24

Earlier quoted context omitted.

Author had LLM help them make a tree of words, and the algo choose which node we're at and offers children as completions. It's clever and cute but, not even close to an LLM.

I mean it's not far off from a super low quant LLM with limited params, like a 1bit quant LLM with low params XD

The trouble with decision trees for language modeling is that they overfit really hard. They don't do the magical generalization that makes LLMs interesting.

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#36

Post by Robin_Message »

dartos wrote: post #33
Robin_Message wrote: post #28

Earlier quoted context omitted.

I think Markov chains have transition probabilities, which this model is lacking. But it's the same idea, just with uniform transition probabilitis.

It seems like it has transition probabilities. Depending on the type of the prior word, it randomly selects the next word from a list of compatible word types. Am I misunderstanding?

It only has probability 0 or 1/n, where n is the number of compatible next words.

There are no numbers in https://github.com/adamjgrant/Tiny-Predictive-Text/blob/main...

A Markov chain could express probabilities like completing "the original" to -> "poster" (0.1), -> "McCoy" (0.2), -> "and best" (0.7) which I don't think this does. But I am tired and maybe also misunderstanding.

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#37

Post by dartos »

Robin_Message wrote: post #36
dartos wrote: post #33

Earlier quoted context omitted.

It seems like it has transition probabilities. Depending on the type of the prior word, it randomly selects the next word from a list of compatible word types. Am I misunderstanding?

It only has probability 0 or 1/n, where n is the number of compatible next words. There are no numbers in https://github.com/adamjgrant/Tiny-Predictive-Text/blob/main... A Markov chain could express probabilities like completing "the original" to -> "poster" (0.1), -> "McCoy" (0.2), -> "and best" (0.7) which I don't think this does. But I am tired and maybe also misunderstanding.

Is a Markov chain where every state has an equal probability not a Markov chain?

Kind of splitting hairs here I guess, but I genuinely don’t know.

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#38

Post by efilife »

mromanuk wrote: post #20
sodimel wrote: post #14

This is cool. I don't want a 1GB+ download and an entire LLM running on my machine (or worse, on someone else machine) to find words. What I really want is some simple predictive text engine to make writing in English easier (because its not my first language), helping me find more complex words that I don't use because I don't know them well enough.

I've developed a neural network model for text correction, prediction, and autocompletion, specifically optimized for one of my iOS keyboard apps. The model is compact, at only 32MB, allowing for swift loading times. It predicts the next word based on the last four to five words, demonstrating robustness against errors, typos, and making minor grammar corrections. The goal was to mimic the functionality of the stock…

Cool. But do you intend on sharing it? Or you just wanted to tell people that you made it?

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#39

Post by qingcharles »

valval wrote: post #12

What's the point? I tested it out for around 10 seconds and noticed that it's completely useless.

I'm not sure why you're getting downvoted, the actual results were pretty subpar in my experience.

The idea is good, though.

Re: Show HN: Predictive text using only 13kb of JavaScript. no LLM

#40

Post by hidelooktropic »

k__ wrote: post #17

What does that mean no LLM? Markov chains have been a thing for a long time and predictive text was used before LLMs.

It means it doesn't use a large language model. I don't think this is a claim to have been the first to do it.
Post reply on HN