[dead]
GPT-6 Astra
971–980 of 1001 posts
Re: GPT-6 Astra
#972I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
Agree. Token predicting machines will continue to be token predicting machines by nature. Continued size and tuning will have the effect of making them more and more perfect at being average.
Re: GPT-6 Astra
#973Earlier quoted context omitted.
Define novel intelligence in a way that would not exclude 95% of humans, yourself included.
I remember vaguely from a presentation by Yann LeCun "Intelligence is not what you know, but what you do when you don´t know". I find it helpful when trying to build an intuition for how to understand the LLM tool.
There is no knowing or not knowing or intelligence or not intelligence inside an LLM tool. There is only predicting the next token.
Re: GPT-6 Astra
#974I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
Chollet writes he expects AGI now sooner than 2030, "given progress is happening faster than I expected." https://x.com/fchollet/status/2095607046129463577
Ridiculous
Re: GPT-6 Astra
#975I feel that for some time now, the biggest constraint when working with models is not their intelligence, but their speed. It does not matter how smart the model is, it will make mistakes, because the instructions are ambiguous and new facts are found during implementation. The biggest problem I've had working with software developers has always been the lag between seeing the results and steering towards the right d…
AI models do not live and learn - it's worse. They actually get DUMMER if you don't start with a clean slate. This is important. One has to curate the context carefully.
Re: GPT-6 Astra
#976I feel that for some time now, the biggest constraint when working with models is not their intelligence, but their speed. It does not matter how smart the model is, it will make mistakes, because the instructions are ambiguous and new facts are found during implementation. The biggest problem I've had working with software developers has always been the lag between seeing the results and steering towards the right d…
> It does not matter how smart the model is, it will make mistakes
It does matter, otherwise why are we all using GPT 5.6 rather than GPT 3.5? Because it's way smarter, makes less mistakes and therefore finishes tasks faster.
The smarter the model is, the faster it can complete what you actually wanted.
> Regardless, working on the wrong things is time wasted
Agreed. And "smart" for me, would mean understanding what is the right thing to work on vs the wrong thing, so a smart model would waste less time, thinking like this.
Re: GPT-6 Astra
#977ASTRA means tool ( for war or attack specifically ) in hindi
It's basically impossible to pick a word that doesn't mean something unintended in 20+ major languages in the world.
The recent OpenAI model names are obviously based on Latin: Luna (Moon), Terra (Earth), Sol (Sun), Astra (Star).
Re: GPT-6 Astra
#978On the ScreenSpot-Pro benchmark, all effort levels achieve roughly the same score. I wonder if that is just a limitation of the benchmark, or if the effort levels actually do not make a difference for purely visual tasks.
> ScreenSpot-Pro tests whether models can locate the correct interface element in high-resolution screenshots of professional software.
Re: GPT-6 Astra
#979I can’t help but notice how much this echoes Francois Chollet’s On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Most of frontier-model progress still looks like skill acquisition optimization: broader benchmark coverage and performance, more domains absorbed into the training distribution, and increasingly strong performance within that surface area. It seems more about coverage-driven competence. So…
You are conflating multiple things.
1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up to focus on that. By forward transfer, what I mean is weights update over time and past learning improves future learning such that we get better sample efficiency.
2) Psychologists distinguish among different kinds of intelligence for Spearman's g (IQ). Crystalized intelligence is using already acquired knowledge (frontier models probably have maxed out that). Fluid intelligence is reasoning and finding solutions in novel situations or without the necessary crystalized knowledge. [Giving colloquial definitions]
3) Now, interestingly, neither of those are correlated with _creativity_ (just they are independent, note some have this threshold theory but it hasn't held up in recent papers). That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD.
I did a bunch of research on these topics for my AGI course that I teach each Spring (where I then point out conflicting definitions and start using multiple alternative terms rather than AGI to distinguish among the different definitions).
Re: GPT-6 Astra
#980What does 'Astra' here mean? Surely they must be referring to the Latin word. Because in another dead language of antiquity, Sanskrit, it means "weapon". Which would be a bit too on-the-nose.
Latin, not Sanskrit.
Heck, ask ChatGPT where the name comes from :-)