Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

141–150 of 411 posts

Re: The changing goalposts of AGI and timelines

#141

Post by yorwba »

ronnier wrote: post #74
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…

Yes, of course there are people in China who, when their job puts them in conflict with their ethics, will decide to do something ethical. I can't think of any war-related examples, since it's been a while since China was involved in any big wars, but I like the story of Liu Lipeng, who used to work as an internet censor: https://madeinchinajournal.com/2025/04/03/me-and-my-censor/

Re: The changing goalposts of AGI and timelines

#142

Post by nradov »

crote wrote: post #103
BoxFour wrote: post #55

Earlier quoted context omitted.

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…

> They don’t think private companies should have any veto power over how the government uses some technology they're provided. On the other hand, why should the government have infinite power to override how a business operates? If you're not able to refuse to sell to the government, isn't that basically forced speech and/or forced labor?

If you want to engage on this topic then you should start by reading up on the Defense Production Act of 1950.

https://www.congress.gov/crs-product/R43767

Re: The changing goalposts of AGI and timelines

#143

Post by BoxFour »

whatshisface wrote: post #114
BoxFour wrote: post #91

Earlier quoted context omitted.

Sure, that’s why I said "on its face." This administration is obviously very different than most. I don’t think Anthropic is wrong to include that clause with this particular administration, and I doubt the administration is internally framing the issue the way I did rather than defaulting to simple authoritarian instincts. But a more reasonable administration could raise the same concern, and I think I would agree w…

I don't think it's reasonable to take something the government is supposed to be protecting (right to contract) and turn them into its biggest threat. That's not security, it's letting the night guard raid the museum.

Sure, I said as such:

> Of course, the reaction is wildly out of proportion. A normal response would just be to stop doing business with the company and move on. Labeling them a supply chain risk is an extreme response.

Re: The changing goalposts of AGI and timelines

#144

Post by stratos123 »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

> AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent.

The statements of what "actual researchers" are you relying upon for your "next 30 years" estimate? How do you reconcile them with the sub-10- or even sub-5-years timelines of other AI researchers, like Daniel Kokotajlo[1] or Andrej Karpathy[2]? For that matter, what about polls of AI researchers, which usually obtain a median much shorter than 30 years [3]?

[1] https://x.com/DKokotajlo/status/1991564542103662729

[2] https://x.com/karpathy/status/1980669343479509025

[3] https://80000hours.org/2025/03/when-do-experts-expect-agi-to...

Re: The changing goalposts of AGI and timelines

#145

Post by logicchains »

djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully."

There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.

Re: The changing goalposts of AGI and timelines

#146

Post by nradov »

sheepscreek wrote: post #38
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…

The military has deployed lethal autonomous weapons since at least 1979. LLMs might be useful for certain missions but from a military perspective they're nothing fundamentally new.

https://www.vp4association.com/aircraft-information-2/32-2/m...

Re: The changing goalposts of AGI and timelines

#147

Post by nradov »

BoxFour wrote: post #76
remarkEon wrote: post #66

Earlier quoted context omitted.

Agree, and I think the labeling of them (Anthropic) a supply chain risk was handled poorly and will likely be reverted over time. That being said, I would be nervous if I was in the Pentagon and depended on Anthropic tooling for something, even if that something was unrelated to kinetic operations. How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called,…

> the government will come knocking anyway. Dario has even said something along these lines at one point: As the technology matures, it’s very possible the government either nationalizes or semi-nationalizes companies like Anthropic. That doesn’t seem out of the realm of possibility if they can’t land on a relationship similar to existing defense contractors like Raytheon, where these kinds of discussions obviously d…

If the government wants a frontier LLM for military purposes then they can just put out a tender. Defense contractors like Anduril will bid on it. The end product might be slightly worse than what Anthropic sells but, as my dad used to say, "close enough for government work".

Re: The changing goalposts of AGI and timelines

#148

Post by nradov »

crote wrote: post #107
fancy_pantser wrote: post #93

Earlier quoted context omitted.

It's explicitly illegal in China. A 2017 national intelligence law compels Chinese companies and individuals to cooperate with state intelligence when asked and without and public notice. China has no equivalent of the whistleblower protection that enables resignations with public letters explaining why, protests, open letters with many signatures, etc. Whenever you see "Chinese whistleblower" in the news, you're loo…

Isn't that basically the same as a National Security Letter and its attached gag order in the USA?

Not at all. If you're an employee at a company that receives a National Security Letter then you can just quit if you want to. Unlike in China, the US government can't force you to keep working there to suit their purposes.

Re: The changing goalposts of AGI and timelines

#150

Post by tokai »

logicchains wrote: post #145
djoldman wrote: post #116

Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully." There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.

Turing test is generally misunderstood, much like Schrodinger's cat, it has devolved in to a pop cultural meme. The test is to evaluate if a machine can think. Not if it is intelligent, not if it is human-like. Its dismissed as a useful by most experts in philosophy of mind, AI, language, etc..

Thinking cool and all but not that extraordinary. Even plants does it.

Post reply on HN