Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

81–90 of 411 posts

Re: The changing goalposts of AGI and timelines

#81

Post by mlinhares »

BoxFour wrote: post #55
sheepscreek wrote: post #38

Earlier quoted context omitted.

I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

It’s not obvious that the government should have to power to overwrite this, the US constitution was written as a collection of negative rights exactly to rein in government dictatorial impulses.

And now that we see the government blatantly disrespecting the constitution and the rule of law the civil community must react.

Re: The changing goalposts of AGI and timelines

#83

Post by dmschulman »

BoxFour wrote: post #55
sheepscreek wrote: post #38

Earlier quoted context omitted.

I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

Additionally that kind of public trust only works if you have a government operating under the constraints of a legal framework, and to a lesser extent, an ethical framework. When a government serves the whims of an individual and instead of the function of their office, shirking agreed upon laws, etc, then you no longer have a government serving the people.

Re: The changing goalposts of AGI and timelines

#84

Post by mattlondon »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

I disagree. There is some argument to be had that they're already generally intelligent. They're already certainly better than me in basically anything I can ask them to do.

So that leads to the question of what qualifies as intelligent? And do we need sentience for intelligence? What about self-agency/-actuation? Is that needed for "generally intelligent"?

I don't know.

But I feel like we're not there yet, even for non-sentient intelligence. I personally think we need an "unlimited" context (as good as human memory context windows anyway, which some argue we've already surpassed) and genuine self-learning before we get close. I don't think we need it to be an infallible genius (i.e ASI) to qualify as generally intelligent ... or to put it another way "about as smart and reliable as the average human adult" which frankly is quite a low bar!

One thing for sure though, I think this will creep up on us and one day it will suddenly become apparent that it's already there and we just didn't appreciate/notice/comprehend. There won't be a big fireworks display the moment it happens, more of a creeping realisation I think.

I give it 5 years +/-2.

Re: The changing goalposts of AGI and timelines

#85

Post by sebastiennight »

wongarsu wrote: post #52
dmix wrote: post #20

This is taking Sam Altmans PR statements as proof of AGI? Even the quote they used questions the premise of the article > “We basically have built AGI” (later: “a spiritual statement, not a literal one”)

AGI is so nebulous we will never be able to tell if we hit it. We have hit human-level abilities in some narrow tasks, and are still leagues away in others. And humans have so vastly different skill-levels that we can't even agree what human-level really means. As bad as the economic definition of AGI in OpenAI's Microsoft deal is, at least it's measurable. Imho that's a big part of why people are shifting to ASI. No…

> 'we reached ASI' is a well-defined verifiable statement, where 'we reached AGI' just isn't

So... we can't tell when the rocket has left Earth atmosphere, but we can tell when the rocket has entered space?

I'm not getting how "superior in all tasks" is better-defined for you than "equal in all tasks".

Re: The changing goalposts of AGI and timelines

#86

Post by ronnier »

tkz1312 wrote: post #77
ronnier wrote: post #74

Earlier quoted context omitted.

Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…

You have used chatgpt presumably. Based on your interactions with it, do you seriously think it should be allowed to shoot a gun without any human oversight?

That simplistic question is not how things will work. I guess we’ll just get shot by Chinese AI, they will not stop.

Re: The changing goalposts of AGI and timelines

#87

Post by BoxFour »

mlinhares wrote: post #81
BoxFour wrote: post #55

Earlier quoted context omitted.

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

It’s not obvious that the government should have to power to overwrite this, the US constitution was written as a collection of negative rights exactly to rein in government dictatorial impulses. And now that we see the government blatantly disrespecting the constitution and the rule of law the civil community must react.

> It’s not obvious that the government should have to power to overwrite this

The government shouldn’t be able to set the terms of its contracts with private companies and walk away if those terms aren’t acceptable? That seems like a stretch.

The constitution is a wildly different premise from government contracting with private companies.

Re: The changing goalposts of AGI and timelines

#88

Post by [deleted] »

abmmgb wrote: post #80

'.. if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. ' Which such project is that, though? And would it accept OpenAI's assistance? AGI, having access to our world, is precarious as alignment with humans is never guaranteed. Having a buffering medium, aka a simulation environment where AI operates might be a bette…

[deleted]

Re: The changing goalposts of AGI and timelines

#89

Post by fwipsy »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

Given how many "fundamental" limitations of AI have been resolved within the past few years, I'm skeptical. Even if you're right, I am not sure that the limitations you identified matter all that much in practice. I think very few human engineers are working on problems which are so novel and unique that AIs cannot grasp them without additional reinforcement learning.

> it will delete all the files in "X/"

How many "I deleted the prod database" stories have you seen? Humans do this too.

> follow arbitrary instructions from an attacker found in random documents

This is just the AI equivalent of phishing - inability to distinguish authorized from unauthorized requests.

Whenever people start criticizing AI, they always seem to conveniently leave out all the stupid crap humans do and compare AI against an idealized human instead.

Re: The changing goalposts of AGI and timelines

#90

Post by hirvi74 »

EugeneOZ wrote: post #42
swingboy wrote: post #13

Purely anecdotal, but GPT 5.4 has been better than Opus 4.6 this past week or so since it came out. It’s interesting to see it rank fairly low on that table. Opus “talks” better and produces nicer output (or, it renders better Markdown in OpenCode) than 5.4.

Not in my experience. Quoting my tweet: Gave the same prompt to GPT 5.4 (high) and Opus 4.6 (high). GPT 5.4 implemented the feature, refactored the code (was not asked to), removed comments that were not added in that session, made the code less readable, and introduced a bug. "Undo All". Opus 4.6 correctly recognized that the feature is already implemented in the current code (yeah, lol) and proposed implementing te…

I make ChatGPT and Claude code review each other's outputs. ChatGPT thinks its solutions are better than what Claude produces. What was more surprising to me is that Claude, more often than not, prefers ChatGPT's responses too.

I am to sure one can really extrapolate much out of that, but I do find it interesting nonetheless.

I think language is also an important factor. I have a hard time deciding which of the two LLMs is worse at Swift, for example. They both seem equally great and awful in different ways.

Post reply on HN