Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small Models Have Arrived

calv.info

Post reply on HN
Print view View original

311–320 of 371 posts

Re: Small Models Have Arrived

#311

Post by throwaway63467 »

mathgenius wrote: post #157
throwaway63467 wrote: post #24

I’m kind of cautiously excited for the next five to ten years, with these AI chips becoming incredibly fast and RAM capacities ramping up its in the cards that we’ll have chips like today’s ATMEL microprocessors that fit on a single board computer and can run small models locally, then all our gizmos can have local AI and I can have a truly intelligent home. Of course there will be a huge push to put all of it in the…

Can we stick one of these in something that looks and sounds like HAL 9000 ?

Hard to decide between Star Trek computer voice, Glados or Jarvis, HAL would also be cool.

Re: Small Models Have Arrived

#312

Post by parasti »

theendisney wrote: post #202
swatcoder wrote: post #77

Earlier quoted context omitted.

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words. The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of res…

At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the pro…

I've done this. Ask Claude to permutate a prompt and run Claude Code or a subagent to observe effects until success. Claude will iterate on the prompt faster than I could and likely catch more edge cases, too. It's one of those tasks where the end goal is very clear and the agent just needs to iterate on permutations. Arguably this is a perfect match, too, because prompt engineering is really just text generation, so why not have a text generation machine do it.

Re: Small Models Have Arrived

#313

Post by svantana »

incrudible wrote: post #307

Where are these 180 IQ models? Even Fable needs guidance so it actually produces output I consider on par with what I could have written myself in a domain I am familiar with, and I am certainly not at the frontier in those domains.

The "IQ 180" label seems to be referring to the (human) CTO of their startup.

Re: Small Models Have Arrived

#315

Post by shakna »

riazrizvi wrote: post #115
dgellow wrote: post #70

Earlier quoted context omitted.

Yes LLMs are a beautiful way to compact knowledge. It would be such a cool technology to develop and worked with if it wasn’t linked to such a toxic industry

I think you're just observing ppl in one of these rare instances where enough of them come together because they are motivated. 'Toxic' is the clamoring sound of a crowded room where what gets through to your ears are just the most annoying snippets of incomplete conversations. I dare you to hang out with any actual people here, understand their viewpoint and listen to what they actually have to say in person, within…

By annoying snippets of incomplete conversations, do you mean Musk's clamoring for legalisation of child porn, or Altman's obsession with doomsday in talking about AI, or Zuckerberg's "Aut Zuck Aut Nihil"?

The people involved at the leadership, in this wave, or incredibly toxic individuals. When the leaders behave a certain way, it tends to be reflected by those following after - intentional or not.

Re: Small Models Have Arrived

#317

Post by embedding-shape »

marcelo-earth wrote: post #278
andsoitis wrote: post #226

> One thing a few investors I've talked with have mentioned: "It's weird we're not seeing more consumer AI companies. Why is that?" What would consumer AI company even be? The frontier labs have declared they will eat everything and they have a head start. Best bet would to be a contrarian and build products and services that people actually want or need. Fine to be AI powered or augmented, but consumer companies do…

And almost all consumer software has AI now. > But what if you want to add AI to your product? Well, now you have some real inference costs on every request! Eventually, these companies just lower their costs by using more efficient models. There were consumer companies built on GPT-3.

> There were consumer companies built on GPT-3.

I don't think anyone is doubting this, but were they any good? Remembering back my first interactions with GPT-3, it was fun, but hardly reliable nor "ready for production", besides making cool demos.

Re: Small Models Have Arrived

#318

Post by squidbeak »

nickysielicki wrote: post #128
jimmaswell wrote: post #91

Earlier quoted context omitted.

This idea has failed to pan out time and time again - people have an instinct that hand-crafted finely-tuned specialized AI systems must be optimal, but throwing more scale and compute to something more generally smart always wins out. It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always perfor…

The Bitter Lesson is very popular right now. It seems true right now. It’s having its moment right now. That doesn’t actually mean it’s axiomatically true. Commenter below gets it absolutely correct: stockfish, which runs on your 5 year old phone, is dramatically better at chess than Fable. Like, so much better that it’s not even remotely comparable. The theory of the Bitter Lesson, and it’s only a theory, is that LL…

You're making a conceptual mistake here, comparing a chess tool to its operator. Deterministic tools produce superior results compared to models in many areas, so we allow models to use tooling.

The correct analogy here is Fable as a second tier player assisting a SuperGM in running stockfish, then assessing its output to identify promising variations.

There might be a limit somewhere that prevents the bitter lesson being axiomatic - for instance where simulations for anything can be exhaustive - so that judgement isn't needed any more as an arbiter. But while there are problems sufficiently complex or large to require a breadth models don't currently have, greater scale and compute will continue to convert to better decision making, and the bitter lesson will remain true (true enough).

Re: Small Models Have Arrived

#319

Post by OtherShrezzing »

jimmaswell wrote: post #91
swatcoder wrote: post #77

Earlier quoted context omitted.

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words. The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of res…

This idea has failed to pan out time and time again - people have an instinct that hand-crafted finely-tuned specialized AI systems must be optimal, but throwing more scale and compute to something more generally smart always wins out. It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always perfor…

I think if you start taking cost into account for tasks which models have saturated a long time ago, the bitter lesson stops being applicable.

If your task is "write a function to sort a list of 1mn entries alphabetically", a small coding model will exactly match the performance of a frontier model (it'll 1-shot it, with 100% accuracy). So the only differentiator is cost, and the frontier model will cost orders of magnitude more than the specialised model.

For most intelligence based tasks, you don't (and never have) needed the tool which "performs best at all tasks". You need the cheapest one which performs adequately for your immediate task.

This doesn't mean the bitter lesson is incorrect. At the frontier, it's still correct. It means that it's not applicable at all to lots of tasks.

Post reply on HN