Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fable and the end of the free lunch

dbreunig.com

Post reply on HN
Print view View original

81–90 of 268 posts

Re: Fable and the end of the free lunch

#81

Post by CamperBob2 »

ACCount37 wrote: post #78
CamperBob2 wrote: post #71

Earlier quoted context omitted.

And yet, there's VibeThinker 3B to bring this long-held premise into question (if not to blast it to pieces.) It is practically illiterate by the standards of larger models, yet performs like models 100x its size on mathematical and logical reasoning tasks.

Which are the kinds of tasks computers have been historically quite good at. It's impressive that it does what it does, don't get me wrong. But if you expect it to replace the likes of GPT 5.6 Luna, let alone Sol? Nah.

Computers have historically been good at answering word problems fed to them verbatim?

Re: Fable and the end of the free lunch

#82

Post by camdenreslink »

csullivannet wrote: post #49
blfr wrote: post #39

Earlier quoted context omitted.

How much are you saving once the more expensive model already has all the context loaded and ready to go?

API calls get more expensive, not less, as you've loaded more context. This is exactly when you want to switch to cheaper models.

There is caching to consider. Switching models throws away the cached tokens.

Re: Fable and the end of the free lunch

#83

Post by ksenzee »

a2ff6eeb0 wrote: post #80
a2ff6eeb0 wrote: post #63

Earlier quoted context omitted.

It's going to be the shareholders of the first companies to crack AGI, and make human brains fully irrelevant economically. With the trillions of dollars that's going in through both investment and users, it's going to happen. I don't believe the human brain has fundamental magic that will make this impossible.

For the downvoters: What magic do you think the human brain has that makes it impossible to emulate acceptably?

LLMs are not emulating the human brain. Somebody may well be able to do that someday, but right now nobody is even trying to.

Re: Fable and the end of the free lunch

#85

Post by josephg »

ksenzee wrote: post #83
a2ff6eeb0 wrote: post #80

Earlier quoted context omitted.

For the downvoters: What magic do you think the human brain has that makes it impossible to emulate acceptably?

LLMs are not emulating the human brain. Somebody may well be able to do that someday, but right now nobody is even trying to.

Why would you need brain emulation to get superhuman intelligence?

Re: Fable and the end of the free lunch

#86

Post by ksenzee »

josephg wrote: post #85
ksenzee wrote: post #83

Earlier quoted context omitted.

LLMs are not emulating the human brain. Somebody may well be able to do that someday, but right now nobody is even trying to.

Why would you need brain emulation to get superhuman intelligence?

Are you making a serious argument that superhuman intelligence is a plausible outcome of training LLMs on everything humanity knows so far? Or are you making the generic assertion that AGI is theoretically possible via means other than emulating the human brain? Because the latter is a strawman (nobody has asserted anything to the contrary), and I have seen no evidence at all to support the former.

Re: Fable and the end of the free lunch

#87

Post by selcuka »

jbstack wrote: post #46
gpjanik wrote: post #40

"When Moore’s Law slowed in the mid-2000s" it did not, in fact, slow down in the mid 2000s, or at all. https://ourworldindata.org/data-insights/moores-law-has-accu...

You've selectively quoted the article. The full quote (emphasis added): "When Moore’s Law slowed in the mid-2000s ( specifically, single-threaded performance stagnated ), we suddenly had to think about parallelization, architecture, memory locality, etc." Your link is talking about transistor count. The article is talking about single-threaded performance. Today's CPUs are faster in large part because they have more…

> Your link is talking about transistor count. The article is talking about single-threaded performance.

But Moore's Law has always been about transistor count, not performance.

Re: Fable and the end of the free lunch

#88

Post by danlugo92 »

lossolo wrote: post #62
rmast wrote: post #57

Most of the things I work on are at least security adjacent. At some point chatting with Fable inevitably leads to it thinking about the security related aspects, tripping the safeguards. Maybe Fable can do the same things better than other models, but having to tiptoe around to avoid tripping safeguards makes GPT 5.6 so much easier to work with that I don’t even bother with Fable (or Opus 5) now.

> At some point chatting with Fable inevitably leads to it thinking about the security related aspects, tripping the safeguards. It happens to me all the time with things that have nothing to do with security, Fable spawns a subagent that then adversarially checks the code Fable just wrote and hits guardrails, with zero prompting from me.

No prompting is also prompting, young padawan

Re: Fable and the end of the free lunch

#89

Post by josephg »

ksenzee wrote: post #86
josephg wrote: post #85

Earlier quoted context omitted.

Why would you need brain emulation to get superhuman intelligence?

Are you making a serious argument that superhuman intelligence is a plausible outcome of training LLMs on everything humanity knows so far? Or are you making the generic assertion that AGI is theoretically possible via means other than emulating the human brain? Because the latter is a strawman (nobody has asserted anything to the contrary), and I have seen no evidence at all to support the former.

I think we can compare the human brain and LLMs on a bunch of capabilities today, and see how we compare. By my reckoning:

- LLMs have better long term memory (they know more than any human) and more working memory (LLMs have fast, uniform access to their whole context window).

- LLMs are faster than we are.

- Humans have online learning (we can do simultaneous learning and inference), giving us advantages in many novel tasks.

- We can learn concepts from far less data. And we can manage our mental context more smoothly.

- We seem to have better world models than current models. AI video just doesn't look right, somehow.

I expect that these remaining weaknesses can be overcome without resorting to human brain emulation. I see no reason to think that current LLMs are at the limit of what technology is capable of.

Re: Fable and the end of the free lunch

#90

Post by [deleted] »

selcuka wrote: post #87
jbstack wrote: post #46

Earlier quoted context omitted.

You've selectively quoted the article. The full quote (emphasis added): "When Moore’s Law slowed in the mid-2000s ( specifically, single-threaded performance stagnated ), we suddenly had to think about parallelization, architecture, memory locality, etc." Your link is talking about transistor count. The article is talking about single-threaded performance. Today's CPUs are faster in large part because they have more…

> Your link is talking about transistor count. The article is talking about single-threaded performance. But Moore's Law has always been about transistor count, not performance.

[deleted]
Post reply on HN