Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

91–100 of 411 posts

Re: The changing goalposts of AGI and timelines

#91

Post by BoxFour »

dmschulman wrote: post #83
BoxFour wrote: post #55

Earlier quoted context omitted.

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

Additionally that kind of public trust only works if you have a government operating under the constraints of a legal framework, and to a lesser extent, an ethical framework. When a government serves the whims of an individual and instead of the function of their office, shirking agreed upon laws, etc, then you no longer have a government serving the people.

Sure, that’s why I said "on its face." This administration is obviously very different than most.

I don’t think Anthropic is wrong to include that clause with this particular administration, and I doubt the administration is internally framing the issue the way I did rather than defaulting to simple authoritarian instincts.

But a more reasonable administration could raise the same concern, and I think I would agree with them.

Re: The changing goalposts of AGI and timelines

#92

Post by deadbabe »

When AI can kill at scale, it means no person is too small or too insignificant to not be worth getting hunted down and killed, it will be cheap and easy. Before it used to be that only high value targets would be worth killing. But now even you, a nobody, will be killed. You say or do something the government doesn’t like, it’s over for you.

Re: The changing goalposts of AGI and timelines

#93

Post by fancy_pantser »

ronnier wrote: post #74
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…

It's explicitly illegal in China.

A 2017 national intelligence law compels Chinese companies and individuals to cooperate with state intelligence when asked and without and public notice.

China has no equivalent of the whistleblower protection that enables resignations with public letters explaining why, protests, open letters with many signatures, etc. Whenever you see "Chinese whistleblower" in the news, you're looking at someone who quietly fled the country first and then blew the whistle. Example: https://www.cnn.com/2026/02/27/us/china-nyc-whistleblower-uf...

Re: The changing goalposts of AGI and timelines

#94

Post by spacemanspiff01 »

remarkEon wrote: post #66
BoxFour wrote: post #55

Earlier quoted context omitted.

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

Agree, and I think the labeling of them (Anthropic) a supply chain risk was handled poorly and will likely be reverted over time. That being said, I would be nervous if I was in the Pentagon and depended on Anthropic tooling for something, even if that something was unrelated to kinetic operations. How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called,…

> How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called, can't remember) don't like?

I was thinking that Anthropic would just be providing the models/setup support to run their models in aws gov cloud. They do not have any real insight into what is being asked. Maybe a few engineers have the specific clearances to access and debug the running systems, but that would one or two people who are embedded to debug inference issues - not something that would be analyzed by others in the company.

The whole 'do not use our models for mass surveillance' is at the end of the day an honor system. Companies have no real way of enforcing that clause, or determining that it has been violated. That being said, at least historically, one has been able to trust the government to abide by commercial agreements. The people who work in cleared positions are generally selected for honesty, and ability, willingness to follow rules.

Re: The changing goalposts of AGI and timelines

#95

Post by qwerpy »

user3939382 wrote: post #79
ronnier wrote: post #74

Earlier quoted context omitted.

Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…

Yes we should dispense with ethics so we can win at all costs. Like your point isn’t invalid but what’s the point of restating something akin to the trolley problem but this time, as if the answer is obvious.

We can debate philosophy while our adversaries use any means at their disposal. Or we can invest in different ideas, see what works, and choose the best option.

Re: The changing goalposts of AGI and timelines

#96

Post by hirvi74 »

wongarsu wrote: post #52
dmix wrote: post #20

This is taking Sam Altmans PR statements as proof of AGI? Even the quote they used questions the premise of the article > “We basically have built AGI” (later: “a spiritual statement, not a literal one”)

AGI is so nebulous we will never be able to tell if we hit it. We have hit human-level abilities in some narrow tasks, and are still leagues away in others. And humans have so vastly different skill-levels that we can't even agree what human-level really means. As bad as the economic definition of AGI in OpenAI's Microsoft deal is, at least it's measurable. Imho that's a big part of why people are shifting to ASI. No…

> AGI is so nebulous we will never be able to tell if we hit it.

I completely agree. We can't even measure each other well, let alone machines.

Re: The changing goalposts of AGI and timelines

#97

Post by mattlondon »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

I don't think humans learn any differently than post it notes TBH We call them text books though!

Re: The changing goalposts of AGI and timelines

#98

Post by rvz »

croes wrote: post #5

> Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do > It can be debated whether arena.ai is a suitable metric for AGI, a strong case can probably be made for why it’s not. However, that’s irrelevant, as the spirit of the self-sacrifice clause is to avoid an arms race, and we are clearly in one. No, the spirit is clearly meant for near AGI and we aren’t near AGI

The "I" in AGI stands for IPO.

The "S" stands for Safety.

Re: The changing goalposts of AGI and timelines

#99

Post by pfortuny »

ronnier wrote: post #74
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…

One of the things about slave coups in ancient times was that they really believed there are things more important than life.

Re: The changing goalposts of AGI and timelines

#100

Post by rdiddly »

ozgung wrote: post #28

"Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks." [Wikipedia] One can argue that they have already achieved this. At least for short termed tasks. Humans are still better at organization, collaboration and carrying out very long tasks like managing a project or a company.

You can't say someone has achieved artificial general intelligence for some specific subset of tasks or parameters; it's a contradiction.
Post reply on HN