Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

51–60 of 411 posts

Re: The changing goalposts of AGI and timelines

#51

Post by lich_king »

sheepscreek wrote: post #38
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…

My most straightforward read is that the military simply doesn't want their contractors to have a say in the war doctrine. Raytheon doesn't get to say "you can only bomb the countries we like, and no hitting hospitals or schools". It doesn't necessarily mean the Pentagon wants to bomb hospitals, but they also don't want to lose autonomy.

A less charitable interpretation is that the current doctrine is "China / Russia will build autonomous killbots, so we can't allow a killbot gap".

I'm frankly less concerned about "proper" military uses than I am about the tech bleeding into the sphere of domestic law enforcement, as it inevitably will.

Re: The changing goalposts of AGI and timelines

#52

Post by wongarsu »

dmix wrote: post #20

This is taking Sam Altmans PR statements as proof of AGI? Even the quote they used questions the premise of the article > “We basically have built AGI” (later: “a spiritual statement, not a literal one”)

AGI is so nebulous we will never be able to tell if we hit it. We have hit human-level abilities in some narrow tasks, and are still leagues away in others. And humans have so vastly different skill-levels that we can't even agree what human-level really means. As bad as the economic definition of AGI in OpenAI's Microsoft deal is, at least it's measurable.

Imho that's a big part of why people are shifting to ASI. Not because we reached AGI, but because 'we reached ASI' is a well-defined verifiable statement, where 'we reached AGI' just isn't

Re: The changing goalposts of AGI and timelines

#53

Post by roromainmain »

IshKebab wrote: post #31
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

[flagged]

I’m not sure it makes much sense to call this naive. When the issue is a moral one, conflicts like this are almost inevitable. Ethical convictions often collide with political or institutional decisions. If a company decides to work with the military and someone finds that fundamentally incompatible with their values, stepping away may simply be the only coherent option for them. It’s sometimes just moral consistency. Moral inconsistency can create deep inner conflict and real psychological distress. This being said, many queer people (not only trans people) seem to develop particularly inclusive values, perhaps because they have personally experienced the consequences of harassment, exclusion, or violence. Having gone through that kind of experience can make someone more sensitive to the vulnerability of others.

Re: The changing goalposts of AGI and timelines

#54

Post by nerdsniper »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

> which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you

This is the best summary of an LLM that I’ve ever seen (for laypeople to “get it”) and is the first that accurately describes my experience. I will say, usually the notes passed to the second person are very impressive quality for the topic. But the “2nd person” still rarely has a deep understanding of it.

Re: The changing goalposts of AGI and timelines

#55

Post by BoxFour »

sheepscreek wrote: post #38
kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…

I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided.

On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government might reject that kind of "we know better than you" type of clause.

Of course, the reaction is wildly out of proportion. A normal response would just be to stop doing business with the company and move on. Labeling them a supply chain risk is an extreme response.

Re: The changing goalposts of AGI and timelines

#56

Post by tombert »

dkwmdkfkdk wrote: post #47
throwaw12 wrote: post #14

OpenAI: - we are building Open AI - only if you have more than $10B net worth - we are against using AI for military purposes - except when that case is allowed by government - we are on a mission to help humanity - again, we define humanity as set of people with more than $10B net worth - surrender? - sure, sure, we will, only to people with more than $10B net worth, they can do whatever they want to our models, we…

Anyone can make these silly negations. As if any other big corp is different. And as if your own imaginary big corp (if you took the time and effort to try to achieve, that is) would have behaved differently given enough pressure from investors and share holders. This is all just very naive

Sorry, no, you shouldn't just handwave away bad behavior just because it's common. That's ridiculous.

If big corporations do things that are unethical, they should be called out, even if they're common. Saying "well everyone's doing it", isn't a good excuse to do things that are unethical.

It's not "naive" to point out the lies that OpenAI told to get to the point that they are now. They were claiming to be a non-profit for awhile, they grew in popularity based in part on that early good-will, and now they are a for-profit company looking to IPO into one of the most valuable corporations on the planet. That's a weird thing. That's a thing that seems to be kind of antithetical to their initial purpose. People should point that out.

Re: The changing goalposts of AGI and timelines

#57

Post by herodoturtle »

0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

Agree with your overall point. Curious what you’re basing the “not in the next 30 years” claim on, if you’d care to expand.

Re: The changing goalposts of AGI and timelines

#58

Post by PunchyHamster »

dkwmdkfkdk wrote: post #47
throwaw12 wrote: post #14

OpenAI: - we are building Open AI - only if you have more than $10B net worth - we are against using AI for military purposes - except when that case is allowed by government - we are on a mission to help humanity - again, we define humanity as set of people with more than $10B net worth - surrender? - sure, sure, we will, only to people with more than $10B net worth, they can do whatever they want to our models, we…

Anyone can make these silly negations. As if any other big corp is different. And as if your own imaginary big corp (if you took the time and effort to try to achieve, that is) would have behaved differently given enough pressure from investors and share holders. This is all just very naive

> Anyone can make these silly negations. As if any other big corp is different.

The point you're desperately trying to miss is that most other companies don't put up those moral claims in the first place

Re: The changing goalposts of AGI and timelines

#59

Post by matricks »

A_D_E_P_T wrote: post #40
ozgung wrote: post #28

"Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks." [Wikipedia] One can argue that they have already achieved this. At least for short termed tasks. Humans are still better at organization, collaboration and carrying out very long tasks like managing a project or a company.

> One can argue that they have already achieved this. No, because they're hugely reliant on their training data and can't really move beyond their training data. This is why you haven't seen an explosion of new LLM-aided scientific discoveries, why Suno can't write a song in a new genre (even if you explain it to Suno in detail and give it actual examples,) etc. This should tell you something enormous about (1) their…

> can’t really move beyond their training data

I don’t even think humans can “move beyond” their sensory data. They generalize using it, which is amazing, but they are still limited by it.* So why is this a reasonable standard for non-biological intelligence?

We have compelling evidence that both can learn in unsupervised settings. (I grant one has to wrap a transformer model with a training harness, but how can anyone sincerely consider this as a disqualifier while admitting that an infant cannot raise itself from birth!)

I’m happy to discuss nuance like different architectures (carbon versus silicon, neurons versus ANNs, etc), but the human tendency to move the goalposts is not something to be proud of. We really need to stop doing this.

* Jeff Hawkins describes the brain as relentlessly searching for invariants from its sensory data. It finds patterns in them and generalizes.

Re: The changing goalposts of AGI and timelines

#60

Post by hirvi74 »

kirubakaran wrote: post #3

""" I resigned from OpenAI. I care deeply about the Robotics team and the work we built together. This wasn’t an easy call. AI has an important role in national security. But surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people. I have deep respect for Sam and the team, and I’m…

> This was about principle, not people.

Why do I not believe this at all? Were things truly sunshine and roses at OpenAI up until this Pentagon debacle? Perhaps I am mistaken, but it seemed like the writing was on the wall years ago.

> I have deep respect for Sam and the team

I have even more questions now.

Post reply on HN