I'm using ollama for parsing and categorizing scraped jobs for a local job board dashboard I check everyday.
Ask HN: Is anyone doing anything cool with tiny language models?
111–120 of 356 posts
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#112I've made a tiny ~1m parameter model that can generate random Magic the Gathering cards that is largely based on Karpathy's nanogpt with a few more features added on top.
I don't have a pre-trained model to share but you can make one yourself from the git repo, assuming you have an apple silicon mac.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#113I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png
You realize this is going to cause carriers to allow the number to send more spam, because it looks like engagement. The best thing to do is to report the offending message to 7726 (SPAM) so the carrier can take action. You can also file complaints at the FTC and FCC websites, but that takes a bit more effort.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#114Is there any experiments in a small models that does paraphrasing? I tried hsing some off-the-shelf models, but it didn't go well.
I was thinking of hooking them in RPGs with text-based dialogue, so that a character will say something slightly different every time you speak to them.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#115I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
That's awesome!
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#116I had an LLM create a playlist for me. I’m tired of the bad playlists I get from algorithms, so I made a specific playlist with an Llama2 based on several songs I like. I started with 50, removed any I didn’t like, and added more to fill in the spaces. The small models were pretty good at this. Now I have a decent fixed playlist. It does get “tired” after a few weeks and I need to add more to it. I’ve never been able…
Interesting! I've sadly found more capable models to really fail on music recommendations for me.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#117I'm playing with the idea of identifying logical fallacies stated by live broadcasters.
I'll be very positively impressed if you make this work; I spend all day every day for work trying to make more capable models perform basic reasoning, and often failing :-P
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#118I have a tiny device that listens to conversations between two people or more and constantly tries to declare a "winner"
All computation on device?
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#119Earlier quoted context omitted.
I think there is real potential here, for smart browsing. Have the llm get the page, replace all the ads with kittens, find non-paywall versions if possible and needed, spoof fingerprint data, detect and highlight AI generated drivel, etc. The site would have no way of knowing that it wasn’t touching eyeballs. We might be able to rake back a bit of the web this way.
You probably wouldn't want to run this in real-time on every site as it'll significantly increase the load on your browser, but as long as it's possible to generate adblock filter rules, the fixes can scale to a pretty large audience.
I was thinking running it in my home lab server as a proxy, but yeah, scaling it to the browser would require some pretty strong hardware. Still, maybe in a couple of years it could be mainstream.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#120I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.
Someone mentioned generating millions of (very short) stories with an LLM a few weeks ago: https://news.ycombinator.com/item?id=42577644
They linked to an interactive explorer that nicely shows the diversity of the dataset, and the HF repo links to the GitHub repo that has the code that generated the stories: https://github.com/lennart-finke/simple_stories_generate
So, it seems there are ways to get varied stories.