GPT-6 Astra
881–890 of 1001 posts
Re: GPT-6 Astra
#882Hmm, 61 on ArtificialAnalysis, effectively matching GPT-5.6 and trailing the new Meta model. How is that possible along with the other metrics they shared? Insanely jagged intelligence?
Re: GPT-6 Astra
#883I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
I was looking at your website and wondering if there is a way to have access to the course material/videos?
In particular: Spring 2025 @ UR : CSC 209/409 Seminar on Artificial General Intelligence
Fall 2024 @ UR : CSC 277/477 End-to-End Deep Learning
Spring 2023 @ UR : CSC 266/466 Frontiers in Deep Learning
Spring 2022 @ Cornell Tech : CS 5787 – Deep Learning
Fall 2021 @ RIT : IMGS 684 – Deep Learning for Vision
Re: GPT-6 Astra
#884I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
Define novel intelligence in a way that would not exclude 95% of humans, yourself included.
Re: GPT-6 Astra
#885I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
Re: GPT-6 Astra
#886I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
And those hole-fillings, for all intents and purposes, look to us like novelty, even if much of it was simply overlooked by us, or, perhaps, unable to attain due to time or other constraints.
Now if you want to talk beyond the sphere, let's call it the "novel novel discovery of the unknown unknowns", then you may have a point, and AI may be more limited than humans in discovering the things that we don't know we don't know. Especially the as-yet-unmodelable things i.e. intuition.
Plenty of discovery left just working from first principles, however. Which I cautiously suggest current frontier AI is good enough to model to a significant enough extent that it is useful for discovery.
Re: GPT-6 Astra
#887I think the thing I'm most excited about is the increase in _user prompting_. If I give a poorly constrained/ambiguous prompt, I don't want the model one-shotting assumptions left and right. The demos of Fable/GPT-6 are impressive, but "real AGI" should act more like a collaborator than either a peon or overachiever. It's a tough balance to get right, and although this has been possible to achieve with additional pro…
>>> The demos of Fable/GPT-6 are impressive, but "real AGI" should act more like a collaborator than either a peon or overachiever. I don't really agree. The thing that makes Fable feel like an actual collaborator is its ability to sus out your real intent when you give ambiguous instructions. It's really good at it. I watched some reviews today and came way with the impression that Astra is not better than Sol in th…
In general, I don't like when I have to prompt models to NOT do something. It's probably difficult for the AI companies to get this right, they should understand ambiguity but still not over-do simple instructions.
Re: GPT-6 Astra
#888I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
Good thought piece here "We Are Losing the Ability to Discover What We Didn’t Know to Ask[1]" By Anne-Laure Le Cunff
It keeps playing on my mind as I see people at work follow some predetermined AI workflow to get their jobs done, the art of being curious and exploring around the problem is so important to the really big innovations. Been thinking about how to address this through some of the harnesses we are developing in the knowledge working space.
Re: GPT-6 Astra
#889I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things. 1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up…
Chollet's distinction is useful. High performance on known tasks is not the same thing as efficient adaptation to a novel task. Prior knowledge and training data can buy skill. That is a central point of On the Measure of Intelligence. But it does not follow that current frontier progress is only "coverage-driven competence." That is a hypothesis. It is not a result established by Chollet's framework.
"Overfitting at scale" is also the wrong term. A model that learns broad representations and applies them successfully to unseen examples is generalizing. The relevant concern is whether apparent novelty is actually inside the effective training distribution, not whether the model is "overfit."
There is also an unstated premise here: that adding broad knowledge and skills cannot improve the machinery used for novel problem solving. I do not see a basis for assuming that. Learned representations, abstractions, reasoning patterns, and cross-domain analogies can themselves support transfer to new tasks. Whether this becomes sufficient for general intelligence is an open question with insufficient data. But its a perfectly valid hypothesis right now that, given enough domain knowledge and symbolic reasoning examples, LLM COULD maybe "Grok" AGI at a certain critical threshold.
And ARC-AGI-3 was specifically designed around novel abstract environments that require exploration and adaptation. Astra scores 99.9% with OpenAI's context-preserving Provider Adapter, and ARC reports that Astra constructed compact symbolic models of unfamiliar environments. That does not prove AGI, but it points in that direction more so than the other way around.
Gc roughly maps to acquired knowledge. Gf roughly maps to reasoning in relatively novel situations. Naming those two categories does not tell us whether increasing acquired knowledge and learned abstractions in an AI can improve Gf-like behavior. That causal question is exactly what is disputed.
And "Frontier models probably have maxed out crystallized intelligence" is just obviously wrong, unless you think they have been able to dig up every a scrap of paper with knowledge/information on it in the entire world, AND that there is no more useful knowledge to be generated left in the universe.
And the statement that intelligence and creativity are independent is simply wrong. A meta-analysis of 112 studies and 34k participants found a positive correlation of about r .25 between intelligence and divergent thinking. It also found that using g, Gf, or Gc did not eliminate that relationship. Creative achievement has a smaller but still positive meta-analytic association with intelligence, around r = .16. These are distinct constructs, not independent constructs.
And this is just a bad take: "AIs are terrible at creativity". At best that depends on which creativity, and I think its straight up wrong. On divergent thinking tasks, the operationalization behind every ADHD study you could cite, LLMs score above most humans, with the top humans still ahead. If you means Big-C, paradigm-shifting creativity, that is a different construct and none of the ADHD evidence transfers to it.
And if I where to say what I subjectively feel and see.... I have ABSOLUTELY no idea how people can say that we are not seeing sparks of creativity from AIs already. If a PERSON produced some of the music, solutions or deductions that I have seen AIs do, people would have NO problem celebrating it as extremely creative.
And finally, the ADHD claim is also, at best, overstated and just as often debunked. There is some evidence that higher subclinical ADHD trait scores, often survey studies only, are associated with better performance on some divergent-thinking measures. But a review of 31 studies did not find a consistent creativity advantage for people with clinical ADHD, and it found no evidence of better convergent thinking.
Okay, I’m done… And nobody noticed that I’m not doing my job here.
Re: GPT-6 Astra
#890I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
They do new stuff all the time. Ask your AI to draw a gerbil riding a unicycle around Pluto and you'd get an image that hasn't been there before. If by genuinely new you mean without any help from past culture, do humans do that? For significant pushing the boundaries of knowledge stuff you maybe need different algorithms like AlphaGo move 37 or Alpha Fold protein folding. Though again how often do humans do that?