Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

181–190 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#181

Post by lionkor »

nozzlegear wrote: post #61

I have a small fish script I use to prompt a model to generate three commit messages based off of my current git diff. I'm still playing around with which model comes up with the best messages, but usually I only use it to give me some ideas when my brain isn't working. All the models accomplish that task pretty well. Here's the script: https://github.com/nozzlegear/dotfiles/blob/master/fish-func... And for this chan…

Those commit messages are pretty terrible, please try to come up with actual messages ;)

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#183

Post by kaspermarstal »

afro88 wrote: post #178
kaspermarstal wrote: post #158

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task. The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If…

How accurate are the classifications?

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower.

[1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#184

Post by pk-protect-ai »

antonok wrote: post #77
binarysneaker wrote: post #71

Earlier quoted context omitted.

Maybe it could also send automated petitions to the EU to undo cookie consent legislation, and reverse some of the enshitification.

Ha, I'm not sure the EU is prepared to handle the deluge of petitions that would ensue. On a more serious note, this must be the first time we can quantitatively measure the impact of cookie consent legislation across the web, so maybe there's something to be explored there.

why don't you spam the companies who want your data instead? The sites can simply stop gathering your data, then they will not require to ask for consent ...

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#185

Post by kolinko »

Apple’s on device models are around 3B if I’m nit mistaken, and they developed some nice tech around them that they published, if I’m not mistaken - where they have just one model, but have switchable finetunings of that model so that it can perform different functionalities depending on context.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#186

Post by ian_zcy »

JLCarveth wrote: post #106

I used a small (3b, I think) model plus tesseract.js to perform OCR on an image of a nutritional facts table and output structured JSON.

what are you feed into the model? Image (like product packaging) or Image of Structured Table? I found out that model performs good in general with sturctured table, but fails a lot over images.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#188

Post by donbreo »

kaspermarstal wrote: post #158

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task. The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If…

Requirements: -Windows

Looks like I'm out... Would be great if there was a google apps script alternative. My company gave all devs linux systems and the business team operates on windows. So I always use browser based tech like Gapps script for complex sheet manipulation

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#189

Post by frail_figure »

pk-protect-ai wrote: post #184
antonok wrote: post #77

Earlier quoted context omitted.

Ha, I'm not sure the EU is prepared to handle the deluge of petitions that would ensue. On a more serious note, this must be the first time we can quantitatively measure the impact of cookie consent legislation across the web, so maybe there's something to be explored there.

why don't you spam the companies who want your data instead? The sites can simply stop gathering your data, then they will not require to ask for consent ...

It’s the same comments on HN as always. They think EU setting up rules is somehow worse than companies breaking them. We see how the US is turning out without pesky EU restrictions :)

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#190

Post by vaylian »

cwmoore wrote: post #63

I'm playing with the idea of identifying logical fallacies stated by live broadcasters.

LLMs are notoriously unreliable with mathematics and logic. I wish you the best of luck, because this would nevertheless be an awesome tool to have.
Post reply on HN