Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small Models Have Arrived

calv.info

Post reply on HN
Print view View original

271–280 of 373 posts

Re: Small Models Have Arrived

#271

Post by MASNeo »

swatcoder wrote: post #77
NitpickLawyer wrote: post #10

> But I also think the demand for "fast/cheap/good-enough" models is just about to take off. There's a sort of "revelation" I had in ~early '24 when I used a 7B local model with a library called Guidance (initially out of MS, then the team moved) to create a flow where the model would receive pseudocode for tests, first write the tests, and once I approved then started writing code until the tests passed. This was be…

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words. The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of res…

Sounds like mainframes. But then, these never quite disappeared to the date.

Re: Small Models Have Arrived

#272

Post by MASNeo »

We mostly use specialized small models because larger are to expensive/slow and are prone to hallucinating quite a bit. Not sure how this is a surprise, seems more like a best practice.

Re: Small Models Have Arrived

#273

Post by chillfox »

nonethewiser wrote: post #249
andsoitis wrote: post #226

> One thing a few investors I've talked with have mentioned: "It's weird we're not seeing more consumer AI companies. Why is that?" What would consumer AI company even be? The frontier labs have declared they will eat everything and they have a head start. Best bet would to be a contrarian and build products and services that people actually want or need. Fine to be AI powered or augmented, but consumer companies do…

There is a thin line. Lots of AI powered things are just workflows you could implement with claude code and some skills. Or even just a prompt on the web chat. But I think these big technological swings sometimes take a long time to shake out. Society is still adapting to the internet. I think there is more opportunity for consumer application of AI. Im surprised we dont see more models in game. Small models that do…

The average game dev cycle is about 7 years these days, so expect to wait some more years before you see llms used much in games.

Re: Small Models Have Arrived

#274

Post by imp0cat »

krisoft wrote: post #46
hartator wrote: post #18

I have trouble seeing the points of using less capable models. I just want the smartest, best, and most capable models. It feels smaller models for speed and cost are just transitions towards better hardware allowing the very best model.

And that is why i always carry my groceries with an Antonov An-225 Mriya. Is it really needed? No, but i refuse to compromise on what is(was/will be) the best.

That plane was destroyed by Russia, wasn't it? I believe it was partially disassembled when Russia invaded Ukraine and so it wasn't possible to save it. :(

Re: Small Models Have Arrived

#275

Post by ianjbutler »

nl wrote: post #218
nickysielicki wrote: post #168

Earlier quoted context omitted.

The conclusion of the bitter lesson would be that a large language model trained on chess commentary as well as being trained on millions of chess games would outperform stockfish which is only trained on millions of chess games. There’s no evidence at this point that this is true.

> a large language model trained on chess commentary as well as being trained on millions of chess games would outperform stockfish which is only trained on millions of chess games Not really, if anything it's closer to the opposite. The Bitter Lesson essay literally has this as an example: > These researchers wanted methods based on human input to win and were disappointed when they did not.[1] and > Enormous initia…

> These researchers wanted methods based on human input to win and were disappointed when they did not.[1]

This was/is basically a strawman though. Like maybe "human input winning" was desirable for chess masters but for computer science wonks? Not the point or the disappoint. It's always neats and scruffies fighting about using some kind of recognizable method (logic) instead of magic (ML).

> breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning.

More to OP's point I think: nowadays when someone wants to beat you over the head with the bitter lesson, they aren't as careful to include learning and search. They want to say learning leads to intuition (magic) whereby we can avoid work (logic/search), and maybe argue or assume from there that neats and scruffies is settled. TBF, something like reasoning in latent space does resemble intuition!

But the real lesson is confirmed every time we bother to check, and not very bitter for anyone. Search/learning/logic are ALL always necessary on any sufficiently difficult problems, and hybrids that interleave always outperform everything else. Stockfish being the example in this thread that different camps of absolutists would like to claim, but also all the MCTS examples, evolving examples, and new hybrids all the time. My favorite lately: https://arxiv.org/pdf/2511.08983

Re: Small Models Have Arrived

#277

Post by duskdozer »

oceanplexian wrote: post #100
zamadatix wrote: post #72

Earlier quoted context omitted.

I got a great deal on ~72 TB of NVMe right before storage prices shot up, doesn't make it any less ridiculous that I have it or any more relevant to people talking about building a NAS now. 99% of people, even in tech, do not have the stupid amounts of hardware people like us hobby on.

Most people in the US have a car, and the average new car is $40,000. Hell where I live a middle class consumer will spend double that on a Boat or an RV and think nothing of it. These aren’t elite tech workers. It’s not unfathomable that if a personal, generally intelligent local AI provides enough utility and doesn’t require you to tweak CLI flags millions of Americans would want one.

Most people in the US can't feasibly hold a job, get groceries, or go to the doctor without having a car.

Re: Small Models Have Arrived

#278

Post by marcelo-earth »

andsoitis wrote: post #226

> One thing a few investors I've talked with have mentioned: "It's weird we're not seeing more consumer AI companies. Why is that?" What would consumer AI company even be? The frontier labs have declared they will eat everything and they have a head start. Best bet would to be a contrarian and build products and services that people actually want or need. Fine to be AI powered or augmented, but consumer companies do…

And almost all consumer software has AI now.

> But what if you want to add AI to your product? Well, now you have some real inference costs on every request!

Eventually, these companies just lower their costs by using more efficient models. There were consumer companies built on GPT-3.

Re: Small Models Have Arrived

#279

Post by noodletheworld »

vatsachak wrote: post #210
noodletheworld wrote: post #193

Earlier quoted context omitted.

Is sol better? Yes. Categorically. Anyone who tells you otherwise and that luna is “just as good” does not know what they are talking about. Going from sol to luna is a downgrade. It is not a question, it is a fact. > Is sol actually worth the extra cost? Is a question only you can answer, because it has no generic answer. Right now, for me, being able to use sol is worth the cost, but using it all the time is not. I…

Idk... Luna is great if you generate specs before implementation. Sure a Lexus is better than a used Prius, until you include price

Anyone who can’t tell the difference between driving those two cars isn't actually driving.

You cant just go “oh hey, I guess they're both cars so I’m taking your lexus away, catch a cab its cheaper” and expect people to just hug you be be like “yay, thanks! I still have a job I guess! :party:”

:P

Re: Small Models Have Arrived

#280

Post by nl »

ianjbutler wrote: post #275
nl wrote: post #218

Earlier quoted context omitted.

> a large language model trained on chess commentary as well as being trained on millions of chess games would outperform stockfish which is only trained on millions of chess games Not really, if anything it's closer to the opposite. The Bitter Lesson essay literally has this as an example: > These researchers wanted methods based on human input to win and were disappointed when they did not.[1] and > Enormous initia…

> These researchers wanted methods based on human input to win and were disappointed when they did not.[1] This was/is basically a strawman though. Like maybe "human input winning" was desirable for chess masters but for computer science wonks? Not the point or the disappoint. It's always neats and scruffies fighting about using some kind of recognizable method (logic) instead of magic (ML). > breakthrough progress e…

> This was/is basically a strawman though. Like maybe "human input winning" was desirable for chess masters but for computer science wonks?

Oh no!

The whole field was full of people whose entire career was built around the idea of developing smart priors.

To quote Wikipedia:

> For computer vision in particular, much progress came from manual feature engineering, such as SIFT features, SURF features, HoG features, bags of visual words, etc. It was a minority position in computer vision that features can be learned directly from data

This undersells the change though! David Lowe's reputation as the best image researcher in the world was based on his SIFT patent[1]

This approach worked until 30 September 2012.

That was a bitter day for many, many computer science researchers.

[1] https://en.wikipedia.org/wiki/Scale-invariant_feature_transf...

Post reply on HN