I copied all the text from this post and used an LLM to generate a list of all the ideas. I do the same for other similar HN post .
well, what are the ideas?
Ask HN: Is anyone doing anything cool with tiny language models?
131–140 of 356 posts
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#132Microsoft published a paper on their FLAME model (60M parameters) for Excel formula repair/completion which outperformed much larger models (>100B parameters). https://arxiv.org/abs/2301.13779
That paper is from over a year ago, and it compared against codex-davinci... which was basically GPT-3, from what I understand. Saying >100B makes it sound a lot more impressive than it is in today's context... 100B models today are a lot more capable. The researchers also compared against a couple of other ancient(/irrelevant today), small models that don't give me much insight.
FLAME seems like a fun little model, and 60M is truly tiny compared to other LLMs, but I have no idea how good it is in today's context, and it doesn't seem like they ever released it.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#133I made a shell alias to translate things from French to English, does that count?
function trans
llm "Translate \"$argv\" from French to English please"
end
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#134I have it running on a Raspberry Pi 5 for offline chat and RAG. I wrote this open-source code for it: https://github.com/persys-ai/persys It also does RAG on apps there, like the music player, contacts app and to-do app. I can ask it to recommend similar artists to listen to based on my music library for example or ask it to quiz me on my PDF papers.
Does https://github.com/persys-ai/persys-server run on the rpi? Is that design 3d printable? Or is that for paid users only.
I can publish it no problem. I’ll create a new repo with instructions for the hardware with CAD files.
Designing a new one for the NVIDIA Orin Nano Super so it might take a few days.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#135I simply use it to de-anonymize code that I typed in via Claude Maybe should write a plugin for it (open source): 1. Put in all your work related questions in the plugin, an LLM will make it as an abstract question for you to preview and send it 2. And then get the answer with all the data back E.g. df[“cookie_company_name”] becomes df[“a”] and back
You're using it to anonymize your code, not de-anonymize someone's code. I was confused by your comment until I read the replies and realized that's what you meant to say.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#136I have a small fish script I use to prompt a model to generate three commit messages based off of my current git diff. I'm still playing around with which model comes up with the best messages, but usually I only use it to give me some ideas when my brain isn't working. All the models accomplish that task pretty well. Here's the script: https://github.com/nozzlegear/dotfiles/blob/master/fish-func... And for this chan…
Awesome need to make one for naming variables too haha
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#137I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting. Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performa…
This is so cool thanks for sharing. I can imagine it’s not technically possible (yet?) but it would be cool if this could simply be run as a browser extension rather than running a docker container
It should be possible using native messaging [1] which can call out to an external binary. The 1password extensions use that to communicate with the password manager binary.
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#138I've been working on a self-hosted, low-latency service for small LLM's. It's basically exactly what I would have wanted when I started my previous startup. The goal is for real time applications, where even the network time to access a fast LLM like groq is an issue.
I haven't benchmarked it yet but I'd be happy to hear opinions on it. It's written in C++ (specifically not python), and is designed to be a self-contained microservice based around llama.cpp.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#139Using llama 3.2 as an interface to a robot. If you can get the latency down, it works wonderfully
Would love to see this applied to a FPS bot in unreal engine.
Re: Ask HN: Is anyone doing anything cool with tiny language models?
#140I'm playing with the idea of identifying logical fallacies stated by live broadcasters.
Automation to identify logical/rhetorical fallacies is a long held dream of mine, would love to follow along with this project if it picks up somehow