Would the marketing not land better when the model is actually publicly usable when officially announced?
GPT-6 Astra
911–920 of 1001 posts
Re: GPT-6 Astra
#912I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
They can do new tasks with in-context learning but its obviously limited by context window
But what does it mean in practice? Obviously we humans also have a limited cognitive capacity.
Let me offer a thought experiment: Let's say that tomorrow we discover Atlantis, with a treasure trove of books about their culture and science, written in a dialect of ancient Greek that we know how to start to analyze, but no one can read fluently. And let's say that you are a billionaire really curious about their culture and want to converse with an "Atlantean expert" as soon as possible. Would you invest your money in a "we-hate-ai-slop(tm)" group of researchers who would abhor AI and instead delegate the books to a massive number of human grad students? Or in a small group of researchers who are willing to use AI agents to go over these? Or maybe just open a chat session with GPT-6 yourself immediately? What would most effectively assuage your curiosity?
Re: GPT-6 Astra
#913Finally, OpenAI has a Fable/Mythos class model. 5.6 Sol felt like 5.5 on steroids, probably just a different checkpoint with a lot more RL post training. I wouldn't be surprised if there are some conceptual similarities to the kind of latent reasoning Anthropic sees in claude's J-space, although those aren't the same thing. Recurrent/looped transformers themselves aren't a new concept, but it's interesting to finally…
yeah i'm wondering the same way... especially in light of the 20x debacle (where we found that 20x of Max vs 5x only applies to the 5hr limit, not the weekly limit, whereas OpenAI's 20x actually is 20x overall). Also Opus 5 has been really tough to work with. I can't understand half of what it says, it's just so damn obscure.
I still use Opus 4.8 for a lot of tasks because I can't stand the way it talks.
Re: GPT-6 Astra
#914I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
> That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD.
I think it's hard to define creativity in the context of AI because they seemingly just make up new hyphenated terms for everything. Is that creativity? If not, what about when they do the same thing different ideas in the latent space?
If we say that simply nailing one concept to another isn't creativity, then AIs are incapable of creativity, while the vast majority of humans are incapable of creativity. This is just a long way of saying "0 AIs have creativity, 0.00001% of humans have creativity", and the difference between zero and a very small number is infinity.
Re: GPT-6 Astra
#915I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
Hi thanks for the insight. Do you see a role for Control Systems(i.e. ones analogus to Instrumentation engineering) playing a role to modulate certain parts of continual learning? One very important way we learn are lived experiences, it's like telling memory:this part is more important( for emotional or social utility values), pay attention. Good or bad lived experiences both count. I guess is that a path that practical research is considering?
Re: GPT-6 Astra
#916I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
I think our AI systems are essentially massive Central Executive Networks. But novel ideas (creativity) come from the Default Mode Network.
These are the difference in what Kahneman called System 2&1 thinking and what the ancients called the Ratio and the Intellect.
LLMs are all ratio. They depend on our intellect for guidance.
Re: GPT-6 Astra
#917I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
Agree. Token predicting machines will continue to be token predicting machines by nature. Continued size and tuning will have the effect of making them more and more perfect at being average.
They learned generic concepts like our brain does to optimize for this particular surprisngly perfect task:
You have to be able to respond to a very generic question in a way that the other entity thinks this is good, comprehensive, etc.
You can call us situation predicting machines as well if you want.
But you undermine what the latent space of an LLM is representing.
Re: GPT-6 Astra
#918Why are OpenAI so keen to start calling things AGI?
Isn't there some corporate/legal shenanigans where they become a real non-profit at that point? Or does it just let them cut Microsoft and other investors out?
There's got to be a business reason for it unrelated to the model capabillities.
Re: GPT-6 Astra
#919I feel that for some time now, the biggest constraint when working with models is not their intelligence, but their speed. It does not matter how smart the model is, it will make mistakes, because the instructions are ambiguous and new facts are found during implementation. The biggest problem I've had working with software developers has always been the lag between seeing the results and steering towards the right d…
AI models do not live and learn - it's worse. They actually get DUMMER if you don't start with a clean slate. This is important. One has to curate the context carefully.
Dumber.
Re: GPT-6 Astra
#920That hero video is interesting. A projector and speech. Maybe I'm in the minority here, but I find speech to text / text to speech (but not live audio mode) is quite comfortable and effective for coding now. The speech to text part can be frustrating if your local tts model does not have word match context for coding. Codex desktop does this remotely well but is slow. I've been experimenting with local software for m…
>this could bring us closer to the dream of more natural, social computing What I saw was multiple people living alone in a small box in a warehouse (probably filled with other boxes) with all of their natural, social interactions directed at a wall. I wonder if this is foreshadowing for the future of work, at least it is what work will look like as envisioned by OpenAI.
this observation is insightful and worth calling out. this paints the picture of a future I'm not excited about