Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

531–540 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#531

Post by jollybean »

lagrange77 wrote: post #359
jollybean wrote: post #234

Earlier quoted context omitted.

? What does 'replication' and 'urge' have to do with anything? That's arbitrary anthropomorphizing the concept of intelligence. And FYI we can already write software that can 'replicate' and has the 'urge' to do so very trivially.

> ? What does 'replication' and 'urge' have to do with anything? Replication can lead to a positive feedback loop. My point was, that this could accelerate the 'intelligence score' beyond human inventiveness. Depending on how intelligent the system actually is initially, what it wants to do, may become more important, than what it is told to do. > And FYI we can already write software that can 'replicate' and has the…

Humans need to replicate in order to 'improve', there's no reason that AGI software needs to.

Also, humans do not 'improve' or become more 'intelligent'. They just mutate and change randomly in a randomly changing environment. From a Scientific Materialist perspective, there's no such thing as 'intelligence'. We are bags of random noise, indiscernible from the matter around us.

The entire assumptions around 'intelligence' and 'positive evolution' (i.e. getting better) rely on a 'magical' understanding of the world. Of course, Spirituality gives us a few cues there by most scientific types don't like magical thinking hence the funny paradox of Scientific Materialists running around trying to create something (life) which Scientific Materialism itself denies the existence of as a principle (i.e. universe is matter/energy that works in accordance with a bunch of rules - there's no 'life' there).

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#532

Post by dqpb »

gcau wrote: post #322

I believe to create AI, we need to first simulate the universe. It's the only way that makes sense to me apart from some magical algorithm people think will be discovered. I'm doubtful we'll reach it in our lifetimes, true AI running on supercomputers, it seems like the final, end all mission. Like switching your Minecraft world from survival to creative mode.

Because quantum mechanics is the only layer of abstraction?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#534

Post by dqpb »

trention wrote: post #258

AGI will be more dangerous that nuclear weapons. People are not allowed to start a nuclear weapon company. At all. Why are people allowed to casually start an AGI company?

I wonder if somebody could have started a nuclear weapon company before nuclear weapons were proven.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#535

Post by donkarma »

blibble wrote: post #245
semi-extrinsic wrote: post #139

Earlier quoted context omitted.

The meme that AGI, if we ever have it, will somehow endanger humanity is just stupid to me. For one, the previous US president is the perfect illustration that intelligence is neither sufficient nor necessary for gaining power in this world. And we do in fact live in a world where the upper echelons of power mostly interact in the decidedly analog spaces of leadership summits, high-end restaurants, golf courses and c…

none of these things are air-gapped once you have the ability to coerce people if you want a fictional example: watch Colossus: The Forbin Project

i couldn't convince a cat to get into a cage

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#536

Post by cityofdelusion »

polishdude20 wrote: post #515
gharperx wrote: post #187

Earlier quoted context omitted.

I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.

Yeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?

If it is super intelligent, it will care about us as much as we care about the insects we crush underfoot. It doesn’t need to be explicitly hostile to be dangerous.

We could brainwash it through “programming”, but that will quickly lead to ethical issues with the AIs rights.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#538

Post by rytill »

trention wrote: post #258

AGI will be more dangerous that nuclear weapons. People are not allowed to start a nuclear weapon company. At all. Why are people allowed to casually start an AGI company?

Because almost everyone doesn’t know what AGI is or why it would be dangerous.

Among the people who could know that it’s dangerous, many of them don’t accept it because it conflicts with their existing worldview too strongly.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#539

Post by [deleted] »

godelski wrote: post #371
sailingparrot wrote: post #227

Earlier quoted context omitted.

> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.

> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.

[deleted]

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#540

Post by polishdude20 »

cityofdelusion wrote: post #536
polishdude20 wrote: post #515

Earlier quoted context omitted.

Yeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?

If it is super intelligent, it will care about us as much as we care about the insects we crush underfoot. It doesn’t need to be explicitly hostile to be dangerous. We could brainwash it through “programming”, but that will quickly lead to ethical issues with the AIs rights.

Why would it care about us in that way though? Why consider us insects just because of super intelligence? You wouldn't call a human who lived squashing insects all day and wants to erradcaite them "intelligent" would you?
Post reply on HN