Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

271–280 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#271

Post by docmars »

kalaksi wrote: post #263
GardenLetter27 wrote: post #240

Earlier quoted context omitted.

It's funny that this is even necessary though - that great EU innovation at work.

Tracking, tracking cookies, banners etc. are a choice done by the website. There are browser addons for making it simpler, though. The transparency requirements and consent for collecting all kinds of PII (this is the regulation) actually is a great innovation.

I think I'd rather see cookie notices handled by a browser API with a common UI, where the default is always "No." Provide that common UI in a popover accessed in the address bar, or a side pane in the browser itself.

If a user logs in or does something requiring cookies that would otherwise prevent normal functionality, prompt them with a Permissions box if they haven't already accepted it in the usual (optional) UI.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#272

Post by kalaksi »

docmars wrote: post #271
kalaksi wrote: post #263

Earlier quoted context omitted.

Tracking, tracking cookies, banners etc. are a choice done by the website. There are browser addons for making it simpler, though. The transparency requirements and consent for collecting all kinds of PII (this is the regulation) actually is a great innovation.

I think I'd rather see cookie notices handled by a browser API with a common UI, where the default is always "No." Provide that common UI in a popover accessed in the address bar, or a side pane in the browser itself. If a user logs in or does something requiring cookies that would otherwise prevent normal functionality, prompt them with a Permissions box if they haven't already accepted it in the usual (optional) UI…

Cookies for normal functionality don't require consent anyway.

But yes, I think just about everybody would like the UX you described. But the entities that track you don't want to make it that easy. You probably know of the do-not-track header too.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#273

Post by Tepix »

antonok wrote: post #99
K0balt wrote: post #96

Earlier quoted context omitted.

I think there is real potential here, for smart browsing. Have the llm get the page, replace all the ads with kittens, find non-paywall versions if possible and needed, spoof fingerprint data, detect and highlight AI generated drivel, etc. The site would have no way of knowing that it wasn’t touching eyeballs. We might be able to rake back a bit of the web this way.

You probably wouldn't want to run this in real-time on every site as it'll significantly increase the load on your browser, but as long as it's possible to generate adblock filter rules, the fixes can scale to a pretty large audience.

Depends on your machine and on the LLM. Could be doable.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#274

Post by 11944351 »

spiritplumber wrote: post #83
thetrash wrote: post #65

I programmed my own version of Tic Tac Toe in Godot, using a Llama 3B as the AI opponent. Not for work flow, but figuring out how to beat it is entertaining during moments of boredom.

Number of players: zero U.S. FIRST STRIKE WINNER: NONE USSR FIRST STRIKE WINNER: NONE NATO / WARSAW PACT WINNER: NONE FAR EAST STRATEGY WINNER: NONE US USSR ESCALATION WINNER: NONE MIDDLE EAST WAR WINNER: NONE USSR CHINA ATTACK WINNER: NONE INDIA PAKISTAN WAR WINNER: NONE MEDITERRANEAN WAR WINNER: NONE HONGKONG VARIANT WINNER: NONE Strange game. The only winning move is not to play

[dead]

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#275

Post by pseudosavant »

jedbrooke wrote: post #42
pseudosavant wrote: post #41

Earlier quoted context omitted.

I wonder how big that model is in RAM/disk. I use LLMs for FFMPEG all the time, and I was thinking about training a model on just the FFMPEG CLI arguments. If it was small enough, it could be a package for FFMPEG. e.g. `ffmpeg llm "Convert this MP4 into the latest royalty-free codecs in an MKV."`

the jetbrains models are about 70MB zipped on disk (one model per language)

That is easily small enough to host as a static SPA web app. I was first thinking it would be cool to make a static web app that would run the model locally. You'd make a query and it'd give the FFMPEG commands.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#276

Post by hackergirl88 »

Evidlo wrote: post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

Where was this during the election

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#277

Post by sebazzz »

I built auto-summarization and grouping in an experimental branch of my hobby-retrospective tool: https://github.com/Sebazzz/Return/tree/experiment/ai-integra...

I’m now just wondering if there is any way to build tests on the input+output of the LLM :D

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#278

Post by Tepix »

bashbjorn wrote: post #244

I'm working on a plugin[1] that runs local LLMs from the Godot game engine. The optimal model sizes seem to be 2B-7B ish, since those will run fast enough on most computers. We recommend that people try it out with Gemma 2 2B (but it will work with any model that works with llama.cpp) At those sizes, it's great for generating non-repetitive flavortext for NPCs. No more "I took an arrow to the knee". Models at around…

Cool. Are you aware of good games that use LLMs like this?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#279

Post by sidcool »

beernet wrote: post #221
kaspermarstal wrote: post #183

Earlier quoted context omitted.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower. [1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

> I don't know. HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

Sometimes it's the joy of creation. Utility and optimization come later. It's fun. Like a hobby.
Post reply on HN