Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#231

Post by yawgmoth »

stogot wrote: post #104
celestialcheese wrote: post #90

Earlier quoted context omitted.

Given the source, I'm skeptical it's not just a troll, but found this explanation [0] plausible as to why those vague spam text exists. If true, this trolling helps the spammers warm those phone numbers up. 0 - https://x.com/nikitabier/status/1867029883387580571

Why does STOP work here?

STOP works thanks to the Telephone Consumer Protection Act (“TCPA”), which offers consumers spam protections and senders a framework on how to behave.

(Edit: It's relevant that STOP didn't come from the TCPA itself, but definitely has teeth due to it)

https://www.infobip.com/blog/a-guide-to-global-sms-complianc...

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#232

Post by ms7892 »

sauravpanda wrote: post #150

We are building a framework to run this tiny language model in the web so anyone can access private LLMs in their browser: https://github.com/sauravpanda/BrowserAI . With just three lines of code, you can run Small LLM models inside the browser. We feel this unlocks a ton of potential for businesses so that they can introduce AI without fear of cost and can personalize the experience using AI. Would love your thought…

Sounds cool. Anyway I can help.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#233

Post by relistan »

lnenad wrote: post #205
relistan wrote: post #174

Earlier quoted context omitted.

Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered. I think it’s gonna be hard for an LLM to do that since that context lives outside the code. But maybe it would, if you hook it to e.g. a ticketing system and include relevant tickets so it can grab context. For instance, in your first example,…

I disagree. When you look at the git history in x months you're gonna have a hard time understanding what was done following your example.

By adding more context? I’m not sure who you’re replying to or what your objection is.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#234

Post by rasmus1610 »

beernet wrote: post #221
kaspermarstal wrote: post #183

Earlier quoted context omitted.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower. [1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

Sometimes people just like to build stuff for the sake of it.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#235

Post by rane »

relistan wrote: post #174
nozzlegear wrote: post #61

I have a small fish script I use to prompt a model to generate three commit messages based off of my current git diff. I'm still playing around with which model comes up with the best messages, but usually I only use it to give me some ideas when my brain isn't working. All the models accomplish that task pretty well. Here's the script: https://github.com/nozzlegear/dotfiles/blob/master/fish-func... And for this chan…

Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered. I think it’s gonna be hard for an LLM to do that since that context lives outside the code. But maybe it would, if you hook it to e.g. a ticketing system and include relevant tickets so it can grab context. For instance, in your first example,…

Most of the time you are not able to fit the "Why?" in the summary.

That's what the body of the commit message is for.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#236

Post by jajko »

rasmus1610 wrote: post #234
beernet wrote: post #221

Earlier quoted context omitted.

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

Sometimes people just like to build stuff for the sake of it.

Almost like hackers, doing shit just for the heck of it because they can (mostly)

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#237

Post by c0wb0yc0d3r »

potamic wrote: post #207
Evidlo wrote: post #81

Earlier quoted context omitted.

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

Why MQTT over HTTP for a low volume, small scale integration?

I’m not OP, but I would hazard a guess that those are the tools that OP has at hand.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#238

Post by jbentley1 »

Tiny language models can do a lot if they are fine tuned for a specific task, but IMO a few things are holding them back:

1. Getting the speed gains is hard unless you are able to pay for dedicated GPUs. Some services offer LoRA as serverless but you don't get the same performance for various technical reasons.

2. Lack of talent to actually do the finetuning. Regular engineers can do a lot of LLM implementation, but when it comes to actually performing training it is a scarcer skillset. Most small to medium orgs don't have people who can do it well.

3. Distribution. Sharing finetunes is hard. HuggingFace exists, but discoverability is an issue. It is flooded with random models with no documentation and it isn't easy to find a good oen for your task. Plus, with a good finetune you also need the prompt and possibly parsing code to make it work the way it is intended and the bundling hasn't been worked out well.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#239

Post by corobo »

beernet wrote: post #221
kaspermarstal wrote: post #183

Earlier quoted context omitted.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower. [1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

Real HN in a nutshell: People who don't build stuff telling people who do build stuff that the thing they built is useless :P

It's a hacker forum, let people hack!

If anything have a dig at OP for posting the thread too soon before the parent commenter has had the chance to gather any data, haha

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#240

Post by GardenLetter27 »

antonok wrote: post #68

I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…

It's funny that this is even necessary though - that great EU innovation at work.
Post reply on HN