Earlier quoted context omitted.
When it can start wars over resources.
So now? Trump generated his tariff list with ChatGPT
On its own.
Evolving OpenAI's Structure
711–720 of 746 posts
Re: Evolving OpenAI's Structure
#712Earlier quoted context omitted.
I think there's a useful distinction that's often missed between AGI and artificial consciousness. We could conceivably have some version of AI that reliably performs any task you throw at it consistently with peak human capabilities, given sufficient tools or hardware to complete whatever that task may be, but lacks subjective experience or independent agency; I would call that AGI. The two concepts have historicall…
One sci-fi example could be based on the replicators from Star Trek, who are able to synthesize any meals on demand. It is not hard to imagine a "cooking robot" as a black box that — given the appropriate ingredients — would cook any dish for you. Press a button, say what you want, and out it comes. Internally, the machine would need to perform lots of tasks that we usually associate with intelligence, from managing…
I immediately thought of Star Trek too, I think the ship's computer was another example of unconscious intelligence. It was incredibly capable and could answer just about any request that anyone made of it. But it had no initiative or motivation of its own.
Re: Evolving OpenAI's Structure
#713Earlier quoted context omitted.
I agree with you I wonder, do you have a hypothesis as to what would be a measurement that would differentiate AGI vs Not-AGI?
It's not much help but when I read "AGI" I picture a fish tank with brains floating in it.
Interesting but I’m not sure very instructive
Re: Evolving OpenAI's Structure
#714Earlier quoted context omitted.
I agree with you I wonder, do you have a hypothesis as to what would be a measurement that would differentiate AGI vs Not-AGI?
Differentiating between AGI and non-AGI, if we ever get remotely close, would be challenging, but for now it's trivial. The defining feature of AGI is recursive self improvement across any field. Without self improvement, you're just regurgitating. Humanity started with no advanced knowledge or even a language. In what should practically be a heartbeat at the speed of distributed computing with perfect memory and com…
So then it’s something exponentially more capable than the most capable human?
> So one fundamental difference is that AGI would not need some absurdly massive data dump to become intelligent.
The first 22 years of life for a “western professional adult” is literally dedicated to a giant bootstrapping info dump
Re: Evolving OpenAI's Structure
#715Earlier quoted context omitted.
Isn't the question you're posing basically Pascals wager? I think the chance they're going to create a "superintelligence" is extremely small. That said I'm sure we're going to have a lot of useful intelligence. But nothing general or self-conscious or powerful enough to be threatening for many decades or even ever. > Predicting the future is famously difficult That's very true, but that fact unfortunately can never…
> I think the chance they're going to create a "superintelligence" is extremely small. I'd say the chance that we never create a superintelligence is extremely small. You either have to believe that for some reason the human brain achieved the maximum intelligence possible, or that progress on AI will just stop for some reason. Most forecasters on prediction markets are predicting AGI within a decade.
Why are you so sure that progress won't just fizzle out at 1/1000 of the performance we would classify as superintelligence?
> that progress on AI will just stop for some reason
Yeah it might. I mean, I'm not blind and deaf, there's been tremendous progress in AI over the last decade, but there's a long way to go to anything superintelligent. If incremental improvement of the current state of the art won't bring superintelligence, can we be sure the fundamental discoveries required will ever be made? Sometimes important paradigm shifts and discoveries take a hundred years just because nobody made the right connection.
Is it certain that every mystery will be solved eventually?
Re: Evolving OpenAI's Structure
#716Earlier quoted context omitted.
> I think the chance they're going to create a "superintelligence" is extremely small. I'd say the chance that we never create a superintelligence is extremely small. You either have to believe that for some reason the human brain achieved the maximum intelligence possible, or that progress on AI will just stop for some reason. Most forecasters on prediction markets are predicting AGI within a decade.
Why are you so sure that progress won't just fizzle out at 1/1000 of the performance we would classify as superintelligence? > that progress on AI will just stop for some reason Yeah it might. I mean, I'm not blind and deaf, there's been tremendous progress in AI over the last decade, but there's a long way to go to anything superintelligent. If incremental improvement of the current state of the art won't bring supe…
Aren't we already passed 1/1000th of the performance we would classify as superintelligence?
There isn't an official precise definition of superintelligence, but it's usually vaguely defined as smarter than humans. Twice as smart would be sufficient by most definitions. We can be more conservative and say we'll only consider superintelligence achieved when it gets to 10x human intelligence. Under that conservative definition, 1/1000th of the performance of superintelligence would be 1% as smart as a human.
We don't have a great way to compare intelligences. ChatGPT already beats humans on several benchmarks. It does better than college students on college-level questions. One study found it gets higher grades on essays than college students. It's not as good as humans on long, complex reasoning tasks. Overall, I'd say it's smarter than a dumb human in most ways, and smarter than a smart human in a few ways.
I'm not certain we'll ever create superintelligence. I just don't see why you think the odds are "extremely small".
Re: Evolving OpenAI's Structure
#717Earlier quoted context omitted.
Differentiating between AGI and non-AGI, if we ever get remotely close, would be challenging, but for now it's trivial. The defining feature of AGI is recursive self improvement across any field. Without self improvement, you're just regurgitating. Humanity started with no advanced knowledge or even a language. In what should practically be a heartbeat at the speed of distributed computing with perfect memory and com…
So then it’s something exponentially more capable than the most capable human? > So one fundamental difference is that AGI would not need some absurdly massive data dump to become intelligent. The first 22 years of life for a “western professional adult” is literally dedicated to a giant bootstrapping info dump
Your quote is a non sequitur to your question. The reason you want to avoid massive data dumps is because there are guaranteed to be errors and flaws. See things like Alpha Go vs Alpha Go Zero. The former was trained on the entirety of human knowledge, the latter was trained entirely on itself.
The zero training version not only ended up dramatically outperforming the 'expert' version, but reached higher levels of competence exponentially faster. And that should be entirely expected. There were obviously tremendous flaws in our understanding of the game, and training on those flaws resulted in software seemingly permanently handicapping itself.
Minimal expert training also has other benefits. The obvious one is that you don't require anywhere near the material and it also enables one to ensure you're on the right track. Seeing software 'invent' fundamental arithmetic is somewhat easier to verify and follow than it producing a hundred page proof advancing, in a novel way, some esoteric edge theory of mathematics. Presumably it would also require orders of magnitude less operational time to achieve such breakthroughs, especially given the reduction in preexisting state.
Re: Evolving OpenAI's Structure
#718Earlier quoted context omitted.
Why are you so sure that progress won't just fizzle out at 1/1000 of the performance we would classify as superintelligence? > that progress on AI will just stop for some reason Yeah it might. I mean, I'm not blind and deaf, there's been tremendous progress in AI over the last decade, but there's a long way to go to anything superintelligent. If incremental improvement of the current state of the art won't bring supe…
Aren't we already passed 1/1000th of the performance we would classify as superintelligence? There isn't an official precise definition of superintelligence, but it's usually vaguely defined as smarter than humans. Twice as smart would be sufficient by most definitions. We can be more conservative and say we'll only consider superintelligence achieved when it gets to 10x human intelligence. Under that conservative de…
I agree, the 1/1000 ratio was a bit too extreme. Like you said, almost any way that's measured it's probably fair to say chatgpt is already there.
Re: Evolving OpenAI's Structure
#719Earlier quoted context omitted.
> So you don't mind if your economic value drops to zero, with all human labour replaced by machines? This was the fear when the cotton gin was invented. It was the ear when cars were created. The same complaint happened with the introduction of electronic, automated, telephone switchboards. Jobs change. Societies change. Unemployment worldwide, is near the lowest it has ever been. Work will change. Society will even…
Where did all the work-horses go? Why is there barely a fraction of the population there once was? Why did they not adapt and find niches where they had a competitive advantage over cars and machines?
The horses weren't the market the economy is selling to, the people are. Ford figured out that people having both time and money is best for the economy. We'll figure out that having all the production capabilities but none of the market benefits nobody.
Re: Evolving OpenAI's Structure
#720Earlier quoted context omitted.
> What else are you looking for? To begin with, systems that don't tell people to use elmer's glue to keep the cheese from sliding off the pizza, displaying a fundamental lack of understanding of.. everything. At minimum it needs to be able to reliably solve hard, unique, but well-defined problems like a group of the most cohesive intelligent people could. It's certainly not AGI until it can do a better job than the…
Majority of people on earth don't solve hard, unique, but well-defined problems, do we? I dont expect AGI to to solve one of Hilbert's list of problems (yet). Your definition of AGI is a bit too imposing. Saying that I believe you would get answers from an LLM better than most of the answers you would get from an average human. IMHO the trend is obvious and we will see if it stalls or keeps the pace.
I don't mean "hard" in the sense that it can easily solve novel problems that no living human knows how to solve, although any "general" intelligence should certainly be capable of learning and making progress on these just like human would, but without limitations of human memory, attention span, relatively short lifetime, and other human needs.
I mean "hard" in the sense that it can reliably replace the best software developers, civil engineers, lawyers, diagnosticians. Not just in economic sense, but by reliably matching the quality of their work 100% of the time.
It should be capable of methodically and reliably arriving at correct answers without expert intervention. It shouldn't be the case that some people claim that they don't know how to code and the LLM generated an entire project for them, while I can confidently claim that LLMs fall flat on their face almost every time I try to use them for more delicate business logic.