Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#211

Post by sam_lowry_ »

staticautomatic wrote: post #55
psyklic wrote: post #2

JetBrains' local single-line autocomplete model is 0.1B (w/ 1536-token context, ~170 lines of code): https://blog.jetbrains.com/blog/2024/04/04/full-line-code-co... For context, GPT-2-small is 0.124B params (w/ 1024-token context).

Is that why their tab completion is so bad now?

Hm... I wonder what your use case it. I do the modern Enterprise Java and the tab completion is a major time saver.

While interactive AI is all about posing, meditating on the prompt, then trying to fix the outcome, IntelliJ tab completion... shows what it will complete as you type and you Tab when you are 100% OK with the completion, which surprisingly happens 90..99% of the time for me, depending on the project.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#213

Post by sainib »

Evidlo wrote: post #81
RVuRnvbM2e wrote: post #73

Earlier quoted context omitted.

This is fantastic. How have your hooked up a mobile number to the llm?

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

Interested for sure.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#215

Post by addandsubtract »

I use a small model to rename my Linux ISOs. I gave it a custom prompt with examples of how I want the output filenames to be structured and then just feed it files to rename. The output only works 90ish percent of the time, so I wrote a little CLI to iterate through the files and accept / retry / edit the changes the LLM outputs.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#216

Post by kaspermarstal »

indolering wrote: post #206
kaspermarstal wrote: post #183

Earlier quoted context omitted.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower. [1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

Y'all definitely need to cross validate a small number of samples by hand. When I did this kind of research, I would hand validate to at least P < .01.

She and one other researcher has manually classified all 7000 papers as per standard protocol. Perhaps for the next article they will measure how this tool agreed with them against them and include it in the protocol if good enough.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#217

Post by sainib »

tinco wrote: post #201
computers3333 wrote: post #176

https://gophersignal.com – I built GopherSignal! It's a lightweight tool that summarizes Hacker News articles. For example, here’s what it outputs for this very post, "Ask HN: Is anyone doing anything cool with tiny language models?": "A user inquires about the use of tiny language models for interesting applications, such as spam filtering and cookie notice detection. A developer shares their experience with using O…

That's cool, I really like it. One piece of feedback: I am usually more interested in the HN comments than in the original article. If you'd include a link to the comments then I might switch to GopherSignal as a replacement for the HN frontpage. My flow is generally: Look at the title and the amount of upvotes to decide if I'm interested in the article. Then view the comments to see if there's interesting discussion…

Agreed..great suggestions. Id consider switching as well.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#218

Post by sainib »

tinco wrote: post #201
computers3333 wrote: post #176

https://gophersignal.com – I built GopherSignal! It's a lightweight tool that summarizes Hacker News articles. For example, here’s what it outputs for this very post, "Ask HN: Is anyone doing anything cool with tiny language models?": "A user inquires about the use of tiny language models for interesting applications, such as spam filtering and cookie notice detection. A developer shares their experience with using O…

That's cool, I really like it. One piece of feedback: I am usually more interested in the HN comments than in the original article. If you'd include a link to the comments then I might switch to GopherSignal as a replacement for the HN frontpage. My flow is generally: Look at the title and the amount of upvotes to decide if I'm interested in the article. Then view the comments to see if there's interesting discussion…

May be even rate each post on the comments activity level.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#219

Post by mogaal »

I bought a tiny business in Brazil, the database (Excel) I inherited with previous customer data *do not include gender*. I need gender to start my marketing campaigns and learn more about my future customer. I used Gemma-2B and Python to determine gender based on the data and it worked perfect

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#220

Post by Nashooo »

mogaal wrote: post #219

I bought a tiny business in Brazil, the database (Excel) I inherited with previous customer data *do not include gender*. I need gender to start my marketing campaigns and learn more about my future customer. I used Gemma-2B and Python to determine gender based on the data and it worked perfect

How did you verify it worked?
Post reply on HN