11–20 of 1001 posts
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
They said that it's able to solve simple math problems. If it's related to A* then maybe it's trying to find a path to something. The answer to a word problem?
Turns out a quarter of a million dollar paycheck plus whatever they get for letting Microsoft gobble them up was more pressing for the staffers. Unless these staff were the unsigned 5%.
Yup. These people are not well.
Could it be named in reference to something like the A* search algorithm? What if Q stands for Query in transformer attention? How would that type of search translate to transformers? They said that it's able to solve simple math problems. If it's related to A* then maybe it's trying to find a path to something. The answer to a word problem?
Just when I thought the ride was over. The belief in what's in the letter could explain some things like how the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology.
Half the board members have a background in ML.
Why should they be able to make the decision to implode the entire company over this?
Why should they have the ability to give zero transparency or comms to the public on their decision?
OpenAI's governance structure was idiotic and glad to see the board members fired.
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
After all, a big objection to LLMs is they are just "fancy autocomplete" or "stochastic parrots". I don't agree, but LLMs famously are "bad at math", that is you can't really train them to do math unless they've seen previous calculations with similar numbers in their training sets. Training an AI system to truly do arbitrary mathematical logic (think "complex word problems with big numbers", not just normal programming statements) would be a huge breakthrough.
> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.
As an example, take the wolf, goat, and cabbage problem, but change the names of the animals and the situation so that the axioms are intact, but the situation no longer seems similar to the classic problem, and therefore has no representation in the training data. GPT-4 can no longer solve this problem consistently.
If Q* can now solve problems like this reliably, it could represent a breakthrough in LLM’s ability to model the world and extrapolate beyond the training data.
> powerful artificial intelligence discovery that they said could threaten humanity Yup. These people are not well.
We have already seen petroleum companies burn the world for profit
Tech is doing its best to eliminate privacy for profit
What would Sam do or not do, for profit?