Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#191

Post by reeeeee »

I built a platform to monitor LLMs that are given complete freedom in the form of a Docker container bash REPL. Currently the models have been offline for some time because I'm upgrading from a single DELL to a TinyMiniMicro Proxmox cluster to run multiple small LLMs locally.

The bots don't do a lot of interesting stuff though, I plan to add the following functionalities:

- Instead of just resetting every 100 messages, I'm going to provide them with a rolling window of context.

- Instead of only allowing BASH commands, they will be able to also respond with reasoning messages, hopefully to make them a bit smarter.

- Give them a better docker container with more CLI tools such as curl and a working package manager.

If you're interested in seeing the developments, you can subscribe on the platform!

https://lama.garden

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#192

Post by TachyonicBytes »

ceritium wrote: post #179

I am doing nothing, but I was wondering if it would make sense to combine a small LLM and SQLITE to parse date time human expressions. For example, given a human input like "last day of this month", the LLM will generate the following query `SELECT date('now','start of month','+1 month','-1 day');` It is probably super overengineering, considering that pretty good libraries are already doing that on different languag…

What libraries have you seen that do this?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#193

Post by TachyonicBytes »

Mashimo wrote: post #172
mritchie712 wrote: post #29

I used local LLMs via Ollama for generating H1's / marketing copy. 1. Create several different personas 2. Generate a ton of variation using a high temperature 3. Compare the variagtions head-to-head using the LLM to get a win / loss ratio The best ones can be quite good. 0 - https://www.definite.app/blog/overkillm

What is an H1?

Not the OP, but they are "Headers". Probably coming from the tag in html. What outsiders probably call "Headlines".

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#194

Post by whywhywhywhy »

pk-protect-ai wrote: post #184
antonok wrote: post #77

Earlier quoted context omitted.

Ha, I'm not sure the EU is prepared to handle the deluge of petitions that would ensue. On a more serious note, this must be the first time we can quantitatively measure the impact of cookie consent legislation across the web, so maybe there's something to be explored there.

why don't you spam the companies who want your data instead? The sites can simply stop gathering your data, then they will not require to ask for consent ...

Because they have no reason to care about what you think or feel or they wouldn't be doing it in the first place.

Cookie notices just gave them another weapon in the end.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#195

Post by laristine »

Mashimo wrote: post #172
mritchie712 wrote: post #29

I used local LLMs via Ollama for generating H1's / marketing copy. 1. Create several different personas 2. Generate a ton of variation using a high temperature 3. Compare the variagtions head-to-head using the LLM to get a win / loss ratio The best ones can be quite good. 0 - https://www.definite.app/blog/overkillm

What is an H1?

Main heading of an article

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#197

Post by nejsjsjsbsb »

ata_aman wrote: post #134
nejsjsjsbsb wrote: post #121

Earlier quoted context omitted.

Does https://github.com/persys-ai/persys-server run on the rpi? Is that design 3d printable? Or is that for paid users only.

I can publish it no problem. I’ll create a new repo with instructions for the hardware with CAD files. Designing a new one for the NVIDIA Orin Nano Super so it might take a few days.

Up to you! Totally understand if you want to hold something back for a paid option!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#198

Post by Etheryte »

kortilla wrote: post #167
spiritplumber wrote: post #125

Earlier quoted context omitted.

yep. the 55% is over a few years.

Right, but if 55% is avg over the last few years, “buy stock” is going to be correct more than not. https://www.crestmontresearch.com/docs/Stock-Yo-Yo.pdf

I think this is a good highlight of why context and reality checks are incredibly important when doing work like this. At first glance, it might look like 55% is a really good result, but in the previous year, a flat buy every day strategy would've been right 56.7% of the time.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#199

Post by bendews »

deet wrote: post #22

We (avy.ai) are using models in that range to analyze computer activity on-device, in a privacy sensitive way, to help knowledge workers as they go about their day. The local models do things ranging from cleaning up OCR, to summarizing meetings, to estimating the user's current goals and activity, to predicting search terms, to predicting queries and actions that, if run, would help the user accomplish their current…

Is this along the lines of rewind.ai, MSCopilot, screenpipe, or something else entirely?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#200

Post by dewey »

great_psy wrote: post #110
deadbabe wrote: post #91

Earlier quoted context omitted.

I’d love to see that. Could you simulate iMessage?

Yes it’s possible, but it’s not something you can easily scale. I had a similar project a few years back that used OSX automations and Shortcuts and Python to send a message everyday to a friend. It required you to be signed in to iMessage on your MacBook. Than was a send operation, the reading of replies is not something I implemented, but I know there is a file somewhere that holds a history of your recent iMessage…

They are all in a SQLite db on your disk.
Post reply on HN