I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Ask HN: Is anyone doing anything cool with tiny language models?
1–10 of 356 posts
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#2JetBrains' local single-line autocomplete model is 0.1B (w/ 1536-token context, ~170 lines of code): https://blog.jetbrains.com/blog/2024/04/04/full-line-code-co...
For context, GPT-2-small is 0.124B params (w/ 1024-token context).
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#3I simply use it to de-anonymize code that I typed in via Claude
Maybe should write a plugin for it (open source):
1. Put in all your work related questions in the plugin, an LLM will make it as an abstract question for you to preview and send it
2. And then get the answer with all the data back
E.g. df[“cookie_company_name”] becomes df[“a”] and back
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#4I simply use it to de-anonymize code that I typed in via Claude Maybe should write a plugin for it (open source): 1. Put in all your work related questions in the plugin, an LLM will make it as an abstract question for you to preview and send it 2. And then get the answer with all the data back E.g. df[“cookie_company_name”] becomes df[“a”] and back
Could you recommend a tiny language model I could try out locally?
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#5"Comedy Writing With Small Generative Models" by Jamie Brew (Strange Loop 2023)
https://m.youtube.com/watch?v=M2o4f_2L0No
Spend the 45 minutes watching this talk. It is a delight. If you are unsure, wait until the speaker picks up the guitar.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#6"Comedy Writing With Small Generative Models" by Jamie Brew (Strange Loop 2023) https://m.youtube.com/watch?v=M2o4f_2L0No Spend the 45 minutes watching this talk. It is a delight. If you are unsure, wait until the speaker picks up the guitar.
Seconded! This was my favorite talk at Strange Loop (including my own).
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#7I simply use it to de-anonymize code that I typed in via Claude Maybe should write a plugin for it (open source): 1. Put in all your work related questions in the plugin, an LLM will make it as an abstract question for you to preview and send it 2. And then get the answer with all the data back E.g. df[“cookie_company_name”] becomes df[“a”] and back
Could you recommend a tiny language model I could try out locally?
Llama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#8No, but I use llama 3.2 1b and qwen2.5 1.5 as bash oneliner generator, always runnimg in console.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#9Pretty sure they are mostly used as fine tuning targets, rather than as-is.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#10JetBrains' local single-line autocomplete model is 0.1B (w/ 1536-token context, ~170 lines of code): https://blog.jetbrains.com/blog/2024/04/04/full-line-code-co... For context, GPT-2-small is 0.124B params (w/ 1024-token context).
That size is on the edge of something you can train at home