Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

281–290 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#281

Post by kaspermarstal »

beernet wrote: post #221
kaspermarstal wrote: post #183

Earlier quoted context omitted.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower. [1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

I am not going to claim or report any kind of accuracy, especially with such a small model and such a specific, context dependent use case. It is the user’s responsibility to cross validate if it’s accurate enough for their use case and upgrade model or use another approach if not.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#282

Post by jkman »

donbreo wrote: post #188
kaspermarstal wrote: post #158

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task. The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If…

Requirements: -Windows Looks like I'm out... Would be great if there was a google apps script alternative. My company gave all devs linux systems and the business team operates on windows. So I always use browser based tech like Gapps script for complex sheet manipulation

Well it's an excel add-in, how else would it work?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#283

Post by pona-a »

macinjosh wrote: post #257
Lutger wrote: post #252

Earlier quoted context omitted.

It has higher salaries for privileged people like senior engineers. Try making ends meet in a lower class job. And you have (almost) free and universal healthcare in Europa, good food available everywhere, drinking water that doesn't poison you, walkable cities, good public transport, somewhat decent police and a functioning legal system. The list goes on. Does this not impact your quality of life? Do you not care ab…

Touch grass. The US is a big place and is nothing like you seem to think it is. Europe on the other hand can't even manage to defend itself and relies on the US for their sheer existence.

Can you enlighten me of a state where none of parent's points apply? I'd be glad to be educated.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#284

Post by iamnotagenius »

jajko wrote: post #249
iamnotagenius wrote: post #222

Earlier quoted context omitted.

I just run llama-cli with the model. Every time I want some "awk" or "find" trickery, I just ask model. Good for throwaway python scripts too.

Can it do 'sed'? I think one major improvement for folks like me would be human->regex LLM translator, ideally also respecting different flavors/syntax for various languages and tools. This has been a bane of me - I run into requirement to develop some complex regexes maybe every 2-3 years, so I dig deep into specs, work on it, deliver eventually if its even possible, and within few months almost completely forget al…

have not tried yet. any specific query? I can try.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#285

Post by pornel »

GardenLetter27 wrote: post #240
antonok wrote: post #68

I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…

It's funny that this is even necessary though - that great EU innovation at work.

The legislation has been watered down by lobbying of the trillion-dollar tracking industry.

The industry knows ~nobody wants to be tracked, so they don't want to let tracking preferences to be easy to express. They want cookie notices to be annoying to make people associate privacy with a bureaucratic nonsense, and stop demanding to have privacy.

There was P3P spec in 2002: https://www.w3.org/TR/P3P/

It even got decent implementation in Internet Explorer, but Google has been deliberately sending a junk P3P header to bypass it.

It has been tried again with a very simple DNT spec. Support for it (that barely existed anyway) collapsed after Microsoft decided to make Do-Not-Track on by default in Edge.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#287

Post by jbs789 »

kaspermarstal wrote: post #281
beernet wrote: post #221

Earlier quoted context omitted.

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

I am not going to claim or report any kind of accuracy, especially with such a small model and such a specific, context dependent use case. It is the user’s responsibility to cross validate if it’s accurate enough for their use case and upgrade model or use another approach if not.

A user buys a car because it gets them from point A to point B. I get what you’re saying though - we are earlier along the adoption curve for these models and more responsibility sits with the user. Over time the expectations will no doubt increase.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#288

Post by goodklopp »

tinco wrote: post #201
computers3333 wrote: post #176

https://gophersignal.com – I built GopherSignal! It's a lightweight tool that summarizes Hacker News articles. For example, here’s what it outputs for this very post, "Ask HN: Is anyone doing anything cool with tiny language models?": "A user inquires about the use of tiny language models for interesting applications, such as spam filtering and cookie notice detection. A developer shares their experience with using O…

That's cool, I really like it. One piece of feedback: I am usually more interested in the HN comments than in the original article. If you'd include a link to the comments then I might switch to GopherSignal as a replacement for the HN frontpage. My flow is generally: Look at the title and the amount of upvotes to decide if I'm interested in the article. Then view the comments to see if there's interesting discussion…

I would love this feature. Regardless, what you have built is really cool

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#290

Post by sdgsgsfg »

kaspermarstal wrote: post #158

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task. The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If…

[dead]
Post reply on HN