These charters are as useful as new year resolutions.
The changing goalposts of AGI and timelines
101–110 of 411 posts
Re: The changing goalposts of AGI and timelines
#102Earlier quoted context omitted.
You have used chatgpt presumably. Based on your interactions with it, do you seriously think it should be allowed to shoot a gun without any human oversight?
That simplistic question is not how things will work. I guess we’ll just get shot by Chinese AI, they will not stop.
You'd rather get shot by domestic bots first?
Re: The changing goalposts of AGI and timelines
#103Earlier quoted context omitted.
I 100% understand and agree the AI community argument around lethal autonomy. But I am trying to understand this from the perspective of defence & govt. Why is it so business as usual for them? Do they consider this at par with missiles with infra-red/heat sensors for tracking/locking? Where does the definition of lethal autonomy begin and end? Just putting this out there as a point to ponder on. By itself, this may…
I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…
> They don’t think private companies should have any veto power over how the government uses some technology they're provided.
On the other hand, why should the government have infinite power to override how a business operates? If you're not able to refuse to sell to the government, isn't that basically forced speech and/or forced labor?
Re: The changing goalposts of AGI and timelines
#104Mission statements and blog posts are meaningless. Cap tables steer behavior and simultaneously protect interests. Stop forming unions or opining on Hacker News. We need to find a way to get citizens on the cap table in a meaningful way (and not at the very, very, very, very end of the waterfall underneath debt holders, hedge funds, governments, preferred investors). We are building this world for us. As it stands, don't fret about a robot taking your job: just make sure you own one of the robots.
Re: The changing goalposts of AGI and timelines
#105> The impotence of naive idealism in the face of economic incentives A great point. I saw blinding idealism during the early days of GPT era.
This was never idealism, it was much more about gaslighting. Billionaire investors have a playbook where they say something to gaslight people into agreeing with them, and then go to the opposite direction. It is already a pattern. For example, most companies would swear they were decided on cutting carbon emissions, just to forget everything when it comes to build data centers. They say some technology is just for t…
For example, most companies would swear they were decided on cutting carbon emissions, just to forget everything when it comes to build data centers.
This doesn't seem contradictory if you consider that success at AGI will solve the problem of carbon emissions, one way or another. If one data center ultimately replaces a whole medium-sized city of commuters...
Re: The changing goalposts of AGI and timelines
#106Earlier quoted context omitted.
I don't think it's about lethal autonomy specifically as much as it's just about government autonomy period. They don’t think private companies should have any veto power over how the government uses some technology they're provided. On its face that’s not a crazy stance: Governments are meant to represent the public, while private companies obviously aren't. I think it’s somewhat understandable why the government mi…
Agree, and I think the labeling of them (Anthropic) a supply chain risk was handled poorly and will likely be reverted over time. That being said, I would be nervous if I was in the Pentagon and depended on Anthropic tooling for something, even if that something was unrelated to kinetic operations. How do they audit that Anthropic can't alter model outputs for contexts they (the ethics board or whatever it's called,…
A counter-argument here: if a private company knows that its technology may be used for human-not-in-loop targeting/surveillance, and knows that its technology is not yet ready to fulfill that use case without meaningful unintended casualties... does that company have an ethical obligation to contractually delineate its inability to offer that service?
In a version of a trolley problem where you're on a track that will kill innocent people, and you have the opportunity to set up a contract that effectively moves a switch to a track without anyone on it, is it not imperative to flip that switch?
(One might argue that increased reaction times might save service members' lives - but the whole point is that if the autonomous targeting is incorrect, it may just as well lead to increased violence and service member casualties in the aggregate.)
And we're not talking about the ethics board manipulating individual token outputs subtly, which would indeed be a supply chain risk - we're talking about a contractual relationship in which, if a supplier detects use outside of the scope of an agreed contract, it has the contractual right to not provide the service for that novel use, while maintaining support for prior use cases.
The fact that the government would use the threat of supply chain risk to enforce a better contract is unprecedented, and it deteriorates the government's standing as a reliable counterparty in general.
Re: The changing goalposts of AGI and timelines
#107Earlier quoted context omitted.
Do Chinese do this in China? Walk away from companies that will be used for war? I doesn’t seem to be prevalent and instead they try to take every advantage they can to push their country, China, to become the most dominate in the world. They must be elated to watch the world’s premier tech companies protest the American government and refusal to work with them. If I wanted China to be weaker I’d hope that Chinese co…
It's explicitly illegal in China. A 2017 national intelligence law compels Chinese companies and individuals to cooperate with state intelligence when asked and without and public notice. China has no equivalent of the whistleblower protection that enables resignations with public letters explaining why, protests, open letters with many signatures, etc. Whenever you see "Chinese whistleblower" in the news, you're loo…
Isn't that basically the same as a National Security Letter and its attached gag order in the USA?
Re: The changing goalposts of AGI and timelines
#108AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…
> which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you This is the best summary of an LLM that I’ve ever seen (for laypeople to “get it”) and is the first that accurately describes my experience. I w…
A layperson analogy I use is that an LLM is like Dora with a really high IQ - it effectively needs everything reexplained to it, and you can’t give it more than a few seconds of context before it just forgets.
Re: The changing goalposts of AGI and timelines
#109AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…
Given how many "fundamental" limitations of AI have been resolved within the past few years, I'm skeptical. Even if you're right, I am not sure that the limitations you identified matter all that much in practice. I think very few human engineers are working on problems which are so novel and unique that AIs cannot grasp them without additional reinforcement learning. > it will delete all the files in "X/" How many "…
> Given how many "fundamental" limitations of AI have been resolved within the past few years
Eh? Which limitations were solved?
Re: The changing goalposts of AGI and timelines
#110Earlier quoted context omitted.
AGI is so nebulous we will never be able to tell if we hit it. We have hit human-level abilities in some narrow tasks, and are still leagues away in others. And humans have so vastly different skill-levels that we can't even agree what human-level really means. As bad as the economic definition of AGI in OpenAI's Microsoft deal is, at least it's measurable. Imho that's a big part of why people are shifting to ASI. No…
> 'we reached ASI' is a well-defined verifiable statement, where 'we reached AGI' just isn't So... we can't tell when the rocket has left Earth atmosphere, but we can tell when the rocket has entered space? I'm not getting how "superior in all tasks" is better-defined for you than "equal in all tasks".
Because ASI is 'superior to the best human' while AGI is 'matches or surpasses humans'. It's hard to agree on what 'matches humans' actually means. Would matching humans in coding involve the level of the average human (basically no coding ability), the level of a junior (the largest group of professionals), a senior ('someone competent') or of the best human we can find?
Or as the scene from 'I, Robot' goes: Will Smith asks the android: 'Can a robot write a symphony? Can a robot turn a blank canvas into a masterpiece?' and the android simply answers 'Can you?' ASI sidesteps that completely