21–26 of 26 posts
This sounds like adding way too much complexity for something that will likely be covered fully by the next gen of frontier models within a single prompt. It also makes it all opaque and difficult to trace.
A sign of system-level optimization starting to overshadow raw/brute-force scaling of foundational models. My view is that foundational models are indeed statistic parrots, just like humans (humans are worse parrots, but human brain's context window is so small that they often do not recognize how broken was human-intermediated intelligence swarm, but such small context window might be a fundamental feature of so-cal…
yes but from my experience abstracting (at least upward) is something all models really struggle with.
I would argue that the best models are quite away from human intelligence, let alone 10%.
> The phrase "frontier model" is starting to mean two things. One is a checkpoint. The other is a system boundary. LLM-isms aside, I don't think we want this to be the case? An LLM, for all its complexity, is something that can be reasoned about. It's picking the next token, until it hits an EOS. The semantics imposed on those tokens (reasoning ,tool call, etc.) are up to the user('s harness) to decide and act on. Th…
I might be wrong, but strongly suspect that Fable 5 is already something in this shape, considering long time to first token while having normal troughput.
A sign of system-level optimization starting to overshadow raw/brute-force scaling of foundational models. My view is that foundational models are indeed statistic parrots, just like humans (humans are worse parrots, but human brain's context window is so small that they often do not recognize how broken was human-intermediated intelligence swarm, but such small context window might be a fundamental feature of so-cal…
A sign of system-level optimization starting to overshadow raw/brute-force scaling of foundational models. My view is that foundational models are indeed statistic parrots, just like humans (humans are worse parrots, but human brain's context window is so small that they often do not recognize how broken was human-intermediated intelligence swarm, but such small context window might be a fundamental feature of so-cal…
> 2. LLMs can somehow be asked to arbitrarily elevate and lower abstraction level (can be seen as a special form of perspective taking) yes but from my experience abstracting (at least upward) is something all models really struggle with. I would argue that the best models are quite away from human intelligence, let alone 10%.
But in the band of LLMs, human cannot match