Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

511–520 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#511

Post by DethNinja »

HL33tibCe7 wrote: post #220
JoshTko wrote: post #138

Earlier quoted context omitted.

It's akin to nuclear weapons. If you do not develop them, then you'd be subject to the will of the ones that develops them first. So invariably you have to invest in AGI lest, an unsavory group develops it first.

Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.

What makes humans special though? AI will be born on Earth so we will share that common trait with it, but perhaps one should have humility to accept that future doesn’t belong to a weak species like humans.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#512

Post by ignoramous »

ericlewis wrote: post #270
fatherzine wrote: post #251

Earlier quoted context omitted.

OTOH if you & your foes develop them both, then there is a probability asymptotically approaching 1 that the weapons will be used over the next X years. Perhaps the only winning move is indeed not to play?

problem is you don't know if they aren't playing - so you must still work on it.

problem is who knows who is "unsavoury".

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#513

Post by polishdude20 »

otikik wrote: post #222

I'm slightly scared that they'll succeed. But not in the usual "robots will kill us" way. What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends.

Yeah the whole ads industry has been getting better and better at hiding the fact that they are ads. It's come to the point where ads are everywhere hidden and almost subliminal. The next step is to just use mind control on people to just get them to buy your stuff.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#514

Post by nscalf »

enneff wrote: post #290
intelVISA wrote: post #261

Earlier quoted context omitted.

if Carmack's in I'm in. Has he ever been drastically wrong?

The jury is still out on VR and Meta but it hardly seems promising.

You think Meta's VR systems aren't promising? My read on the situation is that they are making incredible progress, and doing so at scale. They're already delivering good revenue growth on a product that no one would say is "done". VR isn't a "build a good product" problem, it's a deep tech/hardware R&D problem before you can even open the door to building a good product.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#515

Post by polishdude20 »

gharperx wrote: post #187
WalterBright wrote: post #153

It will decide our fate in a microsecond: extermination.

I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.

Yeah but what would a sufficiently advanced AGI find "worthy"? Why is it that they wouldn't find it worth keeping us around? What would an AGI value? Whatever we program it to value or optimize for? Can it change its mind? If not, then it's controlled by us right? If it's controlled by us, why would it ever decide to wipe everyone out?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#516

Post by polishdude20 »

HL33tibCe7 wrote: post #182
semi-extrinsic wrote: post #139

Earlier quoted context omitted.

The meme that AGI, if we ever have it, will somehow endanger humanity is just stupid to me. For one, the previous US president is the perfect illustration that intelligence is neither sufficient nor necessary for gaining power in this world. And we do in fact live in a world where the upper echelons of power mostly interact in the decidedly analog spaces of leadership summits, high-end restaurants, golf courses and c…

It sounds like you haven’t really thought through AI safety in any real detail at all. Airgapping and the necessity of human input are absolutely not ways to prevent an AGI gaining access to a system. A true, superintelligent AGI could easily extort (or persuade) those humans. If you think concerns over AGI are “stupid”, you haven’t thought about it enough. It’s a massive display of ignorance. The Computerphile AI sa…

What I don't understand is what motivates this world destroying AGI? Like, it's got motives right? How does it get them? Do we program them in? If we do, is the fear that it won't stop at anything in its way to fulfill its objective? If so, what stops it from removing that objective from its motivation? If it can discover zero days to fulfill its objective, wouldn't it reason about itself and it's human-set motives and just use that zero day to change its own programming?

Like, what stops it from changing its motive to something else? And why would it be any more likely to change its motive to be something detrimental to us?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#517

Post by MichaelCollins »

beeboop wrote: post #492
MichaelCollins wrote: post #314

Earlier quoted context omitted.

Why? Because he also wrote some game engines?

Because he's an extremely analytical person and is extremely data driven. He isn't trying to sell some Ted talk

There's all sorts of accomplished people on this list: https://en.wikipedia.org/wiki/List_of_futurologists

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#518

Post by barrysteve »

smnplk wrote: post #208

I think its a silly idea that consciousness can be produced by computation.

Good! The names we have for algorithms are for us, not computers. Computers are just physical machines, they don't care if the math is called AGI, HLI or whatever. People taking these ideas and names seriously are re-programming themselves, not bringing something new into the world.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#520

Post by barrysteve »

willio58 wrote: post #53
ianceicys wrote: post #28

Artificial general intelligence (AGI) — in other words, systems that could successfully perform any intellectual task that a human can. Not in my lifetime, not in this millennium. Possibly in the year 2,300. Weird way to blow $20 million.

It’s not blowing 20 million if it results in meaningful progress in this area. We have something like 2700 billionaires on this planet. This isn’t even a drop in the bucket for someone like that interested in furthering this research. AGI could quite literally shift any job to automation. This is human-experience changing stuff.

Whatever you automate and is truly useful becomes the next hammer, circular saw or vice grips. The human experience which is being changed, is creating better tools for the next generation.
Post reply on HN