Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

81–90 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#81
post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

This is fantastic. How have your hooked up a mobile number to the llm?

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#82
post #59

Not sure it qualifies, but I've started building an Android app that wraps bergamot[0] (the firefox translation models) to have on-device translation without reliance on google. Bergamot is already used inside firefox, but I wanted translation also outside the browser. [0]: bergamot https://github.com/browsermt/bergamot-translator

Definitely interested!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#83

I programmed my own version of Tic Tac Toe in Godot, using a Llama 3B as the AI opponent. Not for work flow, but figuring out how to beat it is entertaining during moments of boredom.

Number of players: zero

U.S. FIRST STRIKE WINNER: NONE

USSR FIRST STRIKE WINNER: NONE

NATO / WARSAW PACT WINNER: NONE

FAR EAST STRATEGY WINNER: NONE

US USSR ESCALATION WINNER: NONE

MIDDLE EAST WAR WINNER: NONE

USSR CHINA ATTACK WINNER: NONE

INDIA PAKISTAN WAR WINNER: NONE

MEDITERRANEAN WAR WINNER: NONE

HONGKONG VARIANT WINNER: NONE

Strange game. The only winning move is not to play

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#85
post #80
post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

Cool! Do you consider the risk of unintentional (and until some moment, an unknown) subscription to some paid SMS service and how do you mitigate it?

I have to whitelist a conversation before the LLM can respond.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#86

My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?

I'm curious what sort of unethical suggestions it's coming up with haha

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#87

I have a tiny device that listens to conversations between two people or more and constantly tries to declare a "winner"

This reminds me of the antics of streamer DougDoug, who often uses LLM APIs to live-summarize, analyze, or interact with his (often multi-thousand-strong) Twitch chat. Most recently I saw him do a GeoGuessr stream where he had ChatGPT assume the role of a detective who must comb through the thousands of chat messages for clues about where the chat thinks the location is, then synthesizes the clamor into a final guess. Aside from constantly being trolled by people spamming nothing but "Kyoto, Japan" in chat, it occasionaly demonstrated a pretty effective incarnation of "the wisdom of the crowd" and was strikingly accurate at times.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#88
post #63

I'm playing with the idea of identifying logical fallacies stated by live broadcasters.

That's fantastic and I'd love to help

So far not much beyond this list of targets to identify https://en.wikipedia.org/wiki/List_of_fallacies

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#89
post #2

JetBrains' local single-line autocomplete model is 0.1B (w/ 1536-token context, ~170 lines of code): https://blog.jetbrains.com/blog/2024/04/04/full-line-code-co... For context, GPT-2-small is 0.124B params (w/ 1024-token context).

I wonder how big that model is in RAM/disk. I use LLMs for FFMPEG all the time, and I was thinking about training a model on just the FFMPEG CLI arguments. If it was small enough, it could be a package for FFMPEG. e.g. `ffmpeg llm "Convert this MP4 into the latest royalty-free codecs in an MKV."`

That’s a great idea, but I feel like it might be hard to get it to be correct enough

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#90
post #69

I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman. http://files.widloski.com/image10%20(1).png http://files.widloski.com/image11.png

Given the source, I'm skeptical it's not just a troll, but found this explanation [0] plausible as to why those vague spam text exists. If true, this trolling helps the spammers warm those phone numbers up.

0 - https://x.com/nikitabier/status/1867029883387580571

Post reply on HN