Although there are better ways to test, I used a 3B model to speed up replies from my local AI server when testing out an application I was developing. Yes I could have mocked up HTTP replies etc., but in this case the small model let me just plug in and go.
Ask HN: Is anyone doing anything cool with tiny language models?
261–270 of 356 posts
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#262Re: Ask HN: Is anyone doing anything cool with tiny language models?
#263I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…
It's funny that this is even necessary though - that great EU innovation at work.
Tracking, tracking cookies, banners etc. are a choice done by the website. There are browser addons for making it simpler, though.
The transparency requirements and consent for collecting all kinds of PII (this is the regulation) actually is a great innovation.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#264Earlier quoted context omitted.
> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...
Real HN in a nutshell: People who don't build stuff telling people who do build stuff that the thing they built is useless :P It's a hacker forum, let people hack! If anything have a dig at OP for posting the thread too soon before the parent commenter has had the chance to gather any data, haha
Just because you can, doesn't mean you should
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#265I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.
Set temperature to 1.0
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#266Earlier quoted context omitted.
Real HN in a nutshell: People who don't build stuff telling people who do build stuff that the thing they built is useless :P It's a hacker forum, let people hack! If anything have a dig at OP for posting the thread too soon before the parent commenter has had the chance to gather any data, haha
Just because you can, doesn't mean you should
If you're building a dinosaur sanctuary sure
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#267Re: Ask HN: Is anyone doing anything cool with tiny language models?
#268I am moderating a playlists manager to restrict them to a range of genders so it classifies song requests as accepted/rejected.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#269I had an LLM create a playlist for me. I’m tired of the bad playlists I get from algorithms, so I made a specific playlist with an Llama2 based on several songs I like. I started with 50, removed any I didn’t like, and added more to fill in the spaces. The small models were pretty good at this. Now I have a decent fixed playlist. It does get “tired” after a few weeks and I need to add more to it. I’ve never been able…
How about having an LLM create a praylist for you?
Then you could implement Salvation as a Service, where you privately confess your sins to a local LLM, and it continuously prays for your eternal soul, recommends penances, and even recites Hail Marys for you.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#270I have a small fish script I use to prompt a model to generate three commit messages based off of my current git diff. I'm still playing around with which model comes up with the best messages, but usually I only use it to give me some ideas when my brain isn't working. All the models accomplish that task pretty well. Here's the script: https://github.com/nozzlegear/dotfiles/blob/master/fish-func... And for this chan…
Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered. I think it’s gonna be hard for an LLM to do that since that context lives outside the code. But maybe it would, if you hook it to e.g. a ticketing system and include relevant tickets so it can grab context. For instance, in your first example,…
> The commit diff already tells you that.
When you squash a branch you'll have 200+ lines of new code on a new feature. The diff is not a quick way to get a summary of what's happening. You should put the "what" in your commit messages.