Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small Models Have Arrived

calv.info

Post reply on HN
Print view View original

281–290 of 377 posts

Re: Small Models Have Arrived

#281

Post by jmalicki »

applfanboysbgon wrote: post #182
CamperBob2 wrote: post #123

Earlier quoted context omitted.

I don't know much about chess engines; do they still use hand-tuned algorithms, or are they more like AlphaZero, where they learn through self-play to beat any/all possible human contenders? I don't believe DeepBlue was automated to that extent, but it may have been. In the latter case, the chess example would tend to support the Bitter Lesson, rather than refute it. I would also be VERY slow to claim that general-pu…

DeepBlue beat Kasparov with essentially raw compute thrown at alpha-beta search. That does support the premise of the bitter lesson in general. But that does not mean the bitter lesson is correctly being applied here . The point is that even if throwing raw compute at a task is better than careful human-crafted algorithms, it's still task-dependent . The current trend with the people blowing hundreds of billions of d…

> Training a model for a specific task with the same amount of compute will outperform training a general-purpose model with the same amount of compute. This should be common sense, right?

I see arguments like this all of the time in ML. There is a large branch of statistical theory that can cast some doubt on this.

Stein's paradox: when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean squared error) than any method that handles the parameters separately.

This goes deeply against "common sense", but is mathematically proven, empirically proven, etc.

If you are just taking a chess model vs. an LLM, you are right.

But if you want a chess model, a medical diagnosis model, and customer service chat model, at a high level the intuition of Stein's paradox is that one combined model will get you better performance than three task-specific models.

Most practitioners don't get this because what amounts to graduate-level stats theory is often brushed aside, and we keep relearning it the hard way.

https://en.wikipedia.org/wiki/Stein%27s_example

Re: Small Models Have Arrived

#282

Post by defrost »

noodletheworld wrote: post #279
vatsachak wrote: post #210

Earlier quoted context omitted.

Idk... Luna is great if you generate specs before implementation. Sure a Lexus is better than a used Prius, until you include price

Anyone who can’t tell the difference between driving those two cars isn't actually driving. You cant just go “oh hey, I guess they're both cars so I’m taking your lexus away, catch a cab its cheaper” and expect people to just hug you be be like “yay, thanks! I still have a job I guess! :party:” :P

Straw argument.

Obvious to the meanest intellect they can tell the difference between the cars.

Re: Small Models Have Arrived

#283

Post by HyperAI »

NitpickLawyer wrote: post #10

> But I also think the demand for "fast/cheap/good-enough" models is just about to take off. There's a sort of "revelation" I had in ~early '24 when I used a 7B local model with a library called Guidance (initially out of MS, then the team moved) to create a flow where the model would receive pseudocode for tests, first write the tests, and once I approved then started writing code until the tests passed. This was be…

[flagged]

Re: Small Models Have Arrived

#284

Post by DarmokTanagra »

teiferer wrote: post #151

A friend of mine told me earlier today that they had a discussion at work (a coding shop) about "downgrading" to luna from sol for cost reasons and that many were quite unhappy about this because they didn't want inferior tech to be forced upon them. Do they have a point? Is sol actually worth the extra cost? Especially if you ramp up the effort level?

They should try "downgrading" to people who don't need a random number generator to do their job for them.

Re: Small Models Have Arrived

#285

Post by noduerme »

swatcoder wrote: post #77
NitpickLawyer wrote: post #10

> But I also think the demand for "fast/cheap/good-enough" models is just about to take off. There's a sort of "revelation" I had in ~early '24 when I used a 7B local model with a library called Guidance (initially out of MS, then the team moved) to create a flow where the model would receive pseudocode for tests, first write the tests, and once I approved then started writing code until the tests passed. This was be…

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words. The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of res…

Honestly, I've been perfectly fine using OpenClaw and Claude 4.6 for the last six months, and am used to its flaws. In general it doesn't screw up anything major, but I also wouldn't ask it to reason about anything more than a few code files at the same time. Since I'm not interested in it replacing higher level reasoning or architectural design, it's pretty capable of just doing the modifications and features I suggest.

One thing notable is that it makes a difference when I specifically refer to line numbers and variable names when I'm asking it to make changes. Meaning, I also read what it writes and understand what it's doing, and have specific queries where I already know what I'd like to change I add.

Re: Small Models Have Arrived

#286

Post by noodletheworld »

defrost wrote: post #282
noodletheworld wrote: post #279

Earlier quoted context omitted.

Anyone who can’t tell the difference between driving those two cars isn't actually driving. You cant just go “oh hey, I guess they're both cars so I’m taking your lexus away, catch a cab its cheaper” and expect people to just hug you be be like “yay, thanks! I still have a job I guess! :party:” :P

Straw argument. Obvious to the meanest intellect they can tell the difference between the cars.

So can anyone using sol and luna.

Don't complain to me if someone responds using a stupid metaphor that proves the opposite of the point they were trying to make.

Re: Small Models Have Arrived

#288

Post by ianjbutler »

nl wrote: post #280
ianjbutler wrote: post #275

Earlier quoted context omitted.

> These researchers wanted methods based on human input to win and were disappointed when they did not.[1] This was/is basically a strawman though. Like maybe "human input winning" was desirable for chess masters but for computer science wonks? Not the point or the disappoint. It's always neats and scruffies fighting about using some kind of recognizable method (logic) instead of magic (ML). > breakthrough progress e…

> This was/is basically a strawman though. Like maybe "human input winning" was desirable for chess masters but for computer science wonks? Oh no! The whole field was full of people whose entire career was built around the idea of developing smart priors. To quote Wikipedia: > For computer vision in particular, much progress came from manual feature engineering, such as SIFT features, SURF features, HoG features, bag…

Is being wrong/ignorant about whether/how something can be automated the same as having a preference for doing it manually? Maybe so if it's your patent, your thesis I guess..

But as it relates to more/less magic, maybe the more modern lens on this is e.g. https://arxiv.org/html/2505.11581v1 . Is manual feature-engineering more like what you'd evolve, or more like what you'd get from SGD ? Feasibility and performance is always a question, there are others like what is robust, stable, adaptable, predictable, explainable. Maybe the manual-features people were interested in something besides the manual part? Maybe the story isn't so simple, and maybe it's not finished yet.

Re: Small Models Have Arrived

#289

Post by TeMPOraL »

hparadiz wrote: post #138
regularfry wrote: post #132

Earlier quoted context omitted.

There's a non-small contingent who lucked into the periodic games machine upgrade at the right time to snag a {3,4,5}090 rig just before everything exploded. It's a small contingent now but it was less so then. And now those people can add a second card for roughly what that whole system would have cost new originally.

The current supply chain problems will eventually pass.

So will the stock of 3090s, as well as their ability to run contemporary local models. And there will be no supply of newer equivalents of those GPUs, because NVIDIA has since wisened up, and is using the very capabilities you need for local LLMs as market segment differentiator.

Re: Small Models Have Arrived

#290

Post by deaux »

irishcoffee wrote: post #113
xnx wrote: post #83

Earlier quoted context omitted.

> 2x 3090s You could sell those and have enough money to pay for hosted inference for years.

I keep seeing this comment. This is _hacker news_ where, back in the day, people just hacked on things, because it was a hobby. They weren't "moneymaxxing" or desperately trying to be as insanely efficient as possible. They hacked on stuff with a can of surge at 3am because it was fun. Your comment is like a meta comment of "LLMs are generating everything, after a while the ouroboros will eat itself. (Which I agree w…

Back in the day the business backing this platform wasn't incubating companies like Flock (YC S17). I think the increased focus on "moneymaxxing" in the community reflects a similar change by its owners.
Post reply on HN