Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

121–130 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#121

I have it running on a Raspberry Pi 5 for offline chat and RAG. I wrote this open-source code for it: https://github.com/persys-ai/persys It also does RAG on apps there, like the music player, contacts app and to-do app. I can ask it to recommend similar artists to listen to based on my music library for example or ask it to quiz me on my PDF papers.

Does https://github.com/persys-ai/persys-server run on the rpi?

Is that design 3d printable? Or is that for paid users only.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#122

I used local LLMs via Ollama for generating H1's / marketing copy. 1. Create several different personas 2. Generate a ton of variation using a high temperature 3. Compare the variagtions head-to-head using the LLM to get a win / loss ratio The best ones can be quite good. 0 - https://www.definite.app/blog/overkillm

clever name!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#124

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

I'm curious what sort of unethical suggestions it's coming up with haha

so far, mostly buying companies owned/ran by horrible people.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#125

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

Have you backtested this in times when markets were not constantly green? Nearly any strategy is good in the good times.

yep. the 55% is over a few years.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#126
post #102

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

Suggestion: calculate the out-of-sample Sharpe ratio[0] of the suggestions over a reasonable period to gauge how good the model would actually perform in terms of return compared to risks. It is better than vanilla accuracy or related metrics. Source: I'm a financial economist. [0]: https://en.wikipedia.org/wiki/Sharpe_ratio

thank you! that's exactly the sort of thing I don't know.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#127

when i feel like casually listening to something, instead of netflix/hulu/whatever, i'll run a ~3b model (qwen 2.5 or llama 3.2) and generate and audio stream of water cooler office gossip. (when it is up, it runs here: https://water-cooler.jothflee.com ). some of the situations get pretty wild, for the office :)

What prompt are you using for this?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#128
I have this idea that a tiny LM would be good at canonicalizing entered real estate addresses. We currently buy a data set and software from Experian, but it feels like something an LM might be very good at. There are lots of weirdnesses in address entry that regexes have a hard time with. We know the bulk of addresses a user might be entering, unless it's a totally new property, so we should be able to train it on that.
Post reply on HN