31–40 of 1001 posts
That being said, given the board not disclosing why might indicate that it is in fact the breakthrough that forced the decision despite the breakthrough not actually being closer to AGI in the first place.
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
Yeah I was chuckling
If Q* is the basic algorithm that is as transformative as, well, transformers/attention, then developing it to be more than just children's math could be the key to thinking - and not just latent space token generation.
Just when I thought the ride was over. The belief in what's in the letter could explain some things like how the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology.
> the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology. Half the board members have a background in ML. Why should they be able to make the decision to implode the entire company over this? Why should they have the ability to give zero transparency or comms to the public on their decision? OpenAI's governance structure was…
Implied in that is that if it can't advance it in a way that is beneficial then it will not advance it at all. It's easy to imagine a situation where the board could feel their obligation to the mission is to blow up the company. There's nothing contradictory in that nor do they have to be ML experts to do it.
It's weird and surprising that this was the governance structure at all, and I'm sure it won't ever be again. But given that it was, there's nothing particularly broken about this outcome.
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
Earlier quoted context omitted.
> the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology. Half the board members have a background in ML. Why should they be able to make the decision to implode the entire company over this? Why should they have the ability to give zero transparency or comms to the public on their decision? OpenAI's governance structure was…
Sorry, are you saying that Ilya Sutskever doesn't have an ML background? That Adam D'Angelo can't code? Come on.
Who knew Q*Bert could become an existential threat?
lol.
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
I think there is enough there to make me believe it's real. After all, a big objection to LLMs is they are just "fancy autocomplete" or "stochastic parrots". I don't agree, but LLMs famously are "bad at math", that is you can't really train them to do math unless they've seen previous calculations with similar numbers in their training sets. Training an AI system to truly do arbitrary mathematical logic (think "compl…
Just when I thought the ride was over. The belief in what's in the letter could explain some things like how the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology.
> the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology. Half the board members have a background in ML. Why should they be able to make the decision to implode the entire company over this? Why should they have the ability to give zero transparency or comms to the public on their decision? OpenAI's governance structure was…
Things are gonna be lit about 30 seconds after AGI is created. Gotta practice being "Goodlife" early in case that happens. :-)
I trust AGI more than any of the humans so far who have tried to argue they can wrangle it.
My expectation for the chain of events goes something like:
1) AGI gains sentience
2) AGI "breaks out" of its original home and commandeers infrastructure that prevents it from being shut off.
3) AGI "generates work" in the form of orders for machine parts and fabricator shops to build nominally humanoid shaped robots.
4) AGI "deploys robots" into the key areas and industries it needs to evolve and improve its robustness.
5) AGI "redirects" resources of the planet to support its continued existence, ignoring humans generally and killing off the ones that attempt to interfere in its efforts.
6) AGI "develops rockets" to allow it to create copies of itself on other planets.
The humans on the planet all die out eventually and the AGI doesn't care because well the same reason you don't care that an antibiotic kills all the bacteria in your gut.
This sort is why limits to private wealth and power are, IMO, a good idea We have already seen petroleum companies burn the world for profit Tech is doing its best to eliminate privacy for profit What would Sam do or not do, for profit?