Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

321–330 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#321
post #81

Earlier quoted context omitted.

Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.

Why MQTT over HTTP for a low volume, small scale integration?

Good, cheap design that takes care of dead letters vs implementing a failover endpoint that would require extra hardware.

MQTT is plug and play in Python. No more costly than a HTTP server.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#323
post #205

Earlier quoted context omitted.

I disagree. When you look at the git history in x months you're gonna have a hard time understanding what was done following your example.

I disagree. If you look back and all you see are commit messages summarizing the diff, you won't get any meaningful information. Telling me `Changed timeout from 30s to 60s` means nothing, while `Increase timeout for slow requests` gives me an actual idea of why that was done. Even better if you add meaningful messages to the commit body. Take a look at commits from large repositories like the Linux kernel and we can…

I mean you're not op but his comment was saying

> Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered.

Which doesn't include what was done. Your example includes both which is fine. But not including what the commit does in the message is an antipattern imho. Everything else that is added is a bonus.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#324

Earlier quoted context omitted.

when you say fine-tuning skills or talent are scarce, do you have specific skills in mind? perhaps engineering for training models (eg making model parallelism work)? or the more ML type skills of designing experiments, choosing which methods to use, figuring out datasets for training, hyperparam tuning/evaluation, etc?

The technical parts are less common and specialized, like understanding the hyperparameters and all that, but I don't think that is the main problem. Most people don't understand how to build a good dataset or how to evaluate their finetune after training. Some parts of this are solid rules like always use a separate validation set, but the task dependent parts are harder to teach. It's a different problem every time…

Finetuning, as I understand it, is mostly laborious and mostly very boring and exhausting work that is not appealing to many engineers. It can be done by people who have some skills in Python or similar language and who have some background in statistics.

OTOH to build the infra for LLMs there's much more stuff involved and it's really hard to find engineers who have the capacity to be both the researchers and developers at the same time. By "researchers" I mean that they have to have a capacity to be able to read through the numerous academic and industry papers, comprehend the tiniest details, and materialize it into the product through the code. I think that's much harder and scarcer skill to find.

That said, I am not undermining the fine-tuning skill, it's a humongous effort, but I think it's not necessarily the skillset problem.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#325
post #323

Earlier quoted context omitted.

I disagree. If you look back and all you see are commit messages summarizing the diff, you won't get any meaningful information. Telling me `Changed timeout from 30s to 60s` means nothing, while `Increase timeout for slow requests` gives me an actual idea of why that was done. Even better if you add meaningful messages to the commit body. Take a look at commits from large repositories like the Linux kernel and we can…

I mean you're not op but his comment was saying > Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered. Which doesn't include what was done. Your example includes both which is fine. But not including what the commit does in the message is an antipattern imho. Everything else that is added is a b…

Many changes require multiple smaller changes, so this is not always possible.

For me the commit message should tell me the what/why and the diff is the how. It's great to understand if, for example, a change was intentional or a bug.

Many times when searching for the source of a bug I could not tell if the line changed was intentional or a mistake because the commit message was simply repeating what was on the diff. If you say your intention was to add something and the diff shows a subtraction, you can easily tell it was a mistake. Contrived example but I think it demonstrates my point.

This only really works if commits are meaningful though. Most people are careless and half their commits are 'fix this', 'fix again', 'wip', etc. At that point the only place that can contain useful information on the intentions are the pull requests/issues around it.

Take a single commit from the Linux kernel: https://github.com/torvalds/linux/commit/08bd5b7c9a2401faabd... It doesn't tell me "add function X, Y and boolean flag Z". It tells us what/why it was done, and the diff shows us how.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#326
post #282

Earlier quoted context omitted.

Requirements: -Windows Looks like I'm out... Would be great if there was a google apps script alternative. My company gave all devs linux systems and the business team operates on windows. So I always use browser based tech like Gapps script for complex sheet manipulation

Well it's an excel add-in, how else would it work?

Excel add-ins can be written with the Office JS API so that they can run on web as well as desktop for Windows and Mac. But I don't think OP's add-in is possible with that API unless the local model can be run in JS.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#327
post #252

Earlier quoted context omitted.

It has higher salaries for privileged people like senior engineers. Try making ends meet in a lower class job. And you have (almost) free and universal healthcare in Europa, good food available everywhere, drinking water that doesn't poison you, walkable cities, good public transport, somewhat decent police and a functioning legal system. The list goes on. Does this not impact your quality of life? Do you not care ab…

Touch grass. The US is a big place and is nothing like you seem to think it is. Europe on the other hand can't even manage to defend itself and relies on the US for their sheer existence.

[dead]

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#328

Earlier quoted context omitted.

It’s the same comments on HN as always. They think EU setting up rules is somehow worse than companies breaking them. We see how the US is turning out without pesky EU restrictions :)

The US has 3x higher salaries, larger houses and a much higher quality of life? I work as a senior engineer in Europe and make barely $4k net per month... and that's considered a "good" salary!

[dead]

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#329

Earlier quoted context omitted.

chatgpt did a stellar job parsing the "books on hard things" thread from a little while ago. my prompt was: Can you identify all the books here, sorted by a weight which is determined based on a combo of the number of votes the comment has, the number of sub-comments, or the number of repeat mentions. Ideally retain hyperlinks if possible.

could you link the HN thread?

google "hn books on hard things" - https://news.ycombinator.com/item?id=42614722

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#330

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task. The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If…

Tried it out, very cool! Fun to see it chugging on a bunch of rows. Had a weird issue where it would recompute values endlessly when I used it in a table, but I had another table it worked with so not sure what that was about
Post reply on HN