Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

91–100 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#91
post #81

Earlier quoted context omitted.

This is fantastic. How have your hooked up a mobile number to the llm?

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

I’d love to see that. Could you simulate iMessage?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#92
when i feel like casually listening to something, instead of netflix/hulu/whatever, i'll run a ~3b model (qwen 2.5 or llama 3.2) and generate and audio stream of water cooler office gossip. (when it is up, it runs here: https://water-cooler.jothflee.com).

some of the situations get pretty wild, for the office :)

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#93
I don’t know if this counts as tiny but I use llama 3B in prod for summarization (kinda).

Its effective context window is pretty small but I have a much more robust statistical model that handles thematic extraction. The llm is essentially just rewriting ~5-10 sentences into a single paragraph.

I’ve found the less you need the language model to actually do, the less the size/quality of the model actually matters.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#94

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

Have you backtested this in times when markets were not constantly green? Nearly any strategy is good in the good times.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#95
post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

Please tell me you have a blog/archive of these somewhere. This was such a joy to read!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#96
post #68

I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…

Maybe it could also send automated petitions to the EU to undo cookie consent legislation, and reverse some of the enshitification.

I think there is real potential here, for smart browsing. Have the llm get the page, replace all the ads with kittens, find non-paywall versions if possible and needed, spoof fingerprint data, detect and highlight AI generated drivel, etc. The site would have no way of knowing that it wasn’t touching eyeballs. We might be able to rake back a bit of the web this way.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#97

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

[deleted]

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#98
post #81

Earlier quoted context omitted.

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

I’d love to see that. Could you simulate iMessage?

If you mean hook this into iMessage, I don't know. I'm willing to bet it's way harder though because Apple

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#99
post #96

Earlier quoted context omitted.

Maybe it could also send automated petitions to the EU to undo cookie consent legislation, and reverse some of the enshitification.

I think there is real potential here, for smart browsing. Have the llm get the page, replace all the ads with kittens, find non-paywall versions if possible and needed, spoof fingerprint data, detect and highlight AI generated drivel, etc. The site would have no way of knowing that it wasn’t touching eyeballs. We might be able to rake back a bit of the web this way.

You probably wouldn't want to run this in real-time on every site as it'll significantly increase the load on your browser, but as long as it's possible to generate adblock filter rules, the fixes can scale to a pretty large audience.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#100
post #81

Earlier quoted context omitted.

This is fantastic. How have your hooked up a mobile number to the llm?

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

Yes, I'd be interested in that!
Post reply on HN