Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#141

Post by sebastiennight »

binarysneaker wrote: post #71
antonok wrote: post #68

I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…

Maybe it could also send automated petitions to the EU to undo cookie consent legislation, and reverse some of the enshitification.

To me this take is like smokers complaining that the evil government is forcing the good tobacco companies to degrade the experience by adding pictures of cancer patients on cigarette packs.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#143

Post by spiritplumber »

bongodongobob wrote: post #109
spiritplumber wrote: post #78

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

You can literally flip coins and get better than 50% success in a bull market. Just buy index funds and spend your time on something that isn't trying to beat entropy. You won't be able to.

INSUFFICIENT DATA FOR A MEANINGFUL ANSWER.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#144

Post by 4mitkumar »

sidravi1 wrote: post #129

We fine-tuned a Gemma 2B to identify urgent messages sent by new and expecting mothers on a government-run maternal health helpline. https://idinsight.github.io/tech-blog/blog/enhancing_materna...

Such a fun thread but this is the kind of applications that perk up my attention!

Very cool!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#145

Post by deivid »

deivid wrote: post #59

Not sure it qualifies, but I've started building an Android app that wraps bergamot[0] (the firefox translation models) to have on-device translation without reliance on google. Bergamot is already used inside firefox, but I wanted translation also outside the browser. [0]: bergamot https://github.com/browsermt/bergamot-translator

I would be very interested if someone is aware of any small/tiny models to perform OCR, so the app can translate pictures as well

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#146

Post by deivid »

JLCarveth wrote: post #106

I used a small (3b, I think) model plus tesseract.js to perform OCR on an image of a nutritional facts table and output structured JSON.

What was the model? What kind of performance did you get out of it?

Could you share a link to your project, if it is public?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#147

Post by TMWNN »

Uehreka wrote: post #20
behohippy wrote: post #14

I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really…

Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves? Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.

> Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?

> Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff.

100% relevant: "Someday" https://en.wikipedia.org/wiki/Someday_(short_story)> by Isaac Asimov, 1956

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#148

Post by metadat »

Evidlo wrote: post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

I love this, more please!!!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#149

Post by febed »

spiritplumber wrote: post #78

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

What data do you analyze?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#150

Post by sauravpanda »

We are building a framework to run this tiny language model in the web so anyone can access private LLMs in their browser: https://github.com/sauravpanda/BrowserAI.

With just three lines of code, you can run Small LLM models inside the browser. We feel this unlocks a ton of potential for businesses so that they can introduce AI without fear of cost and can personalize the experience using AI.

Would love your thoughts and what we can do more or better!

Post reply on HN