I am, in a way by using EHR/EMR data for fine tuning so agents can query each other for medical records in a HIPPA compliant manner.
Ask HN: Is anyone doing anything cool with tiny language models?
11–20 of 356 posts
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#12Microsoft published a paper on their FLAME model (60M parameters) for Excel formula repair/completion which outperformed much larger models (>100B parameters).
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#13Earlier quoted context omitted.
Could you recommend a tiny language model I could try out locally?
Llama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code
+1 this idea. I do the same. Just do it locally using ollama, also using 3.2 3b
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#14I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#15We're using small language models to detect prompt injection. Not too cool, but at least we can publish some AI-related stuff on the internet without a huge bill.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#16I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
this sounds pretty cool, do you have any video/media of it?
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#17I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
this is so cool, any chance you post a video?
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#18Microsoft published a paper on their FLAME model (60M parameters) for Excel formula repair/completion which outperformed much larger models (>100B parameters). https://arxiv.org/abs/2301.13779
This is really cool. Is this already in Excel?
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#19Kinda? All local so very much personal, non-business use. I made Ollama talk in a specific persona styles with the idea of speaking like Spider Jerusalem, when I feel like retaining some level of privacy by avoiding phrases I would normally use. Uncensored llama just rewrites my post with a specific persona's 'voice'. Works amusingly well for that purpose.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#20I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?
Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.