OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

So: we might be close to a breakthrough, that breakthrough could get out of hand, then it could kill a billion+ people.

If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow).

We also know how to solve complex reasoning chains that require backtracking. Prolog has been around since 1972. It's not used that much because that's not the programming problem that most people are solving.

Why not use a tool for what it's good for and pick different tools for other problems they are better for? LLMs are good for summarization, autocompletion, and as an input to many other language problems like spelling and bigrams. They're not good at math. Computers are really good at math.

There's a theorem that an LLM can compute any computable function. That's true, but so can lambda calculus. We don't program in raw lambda calculus because it's terribly inefficient. Same with LLMs for arithmetic problems.

"... To fly away...?"

> It's intelligence in the sense that jerking your arm away from a hotplate is intelligence, which is to say it's not cognitive reasoning, just genetically hardwired triggers.

Sooo, we agree -- the flies are not very intelligent or more likely not intelligent at all.

Sooo, I tried to erect some borders on intelligence, excluded flies but included solving that geometry problem.

> AGI has been defined by OpenAI as something that can do most economically viable activities better than humans can.

This is the first I heard of their definition. Soooo, I didn't consider their definition.

Of course, NASA does not get to define the speed of light. My local electric utility does not get to define 1000 Watts per hour, a KWH.

The OpenAI definition of AGI is an interesting goal, but the acronym abbreviates artificial general intelligence, and it is not clear that it is appropriate for OpenAI to define intelligence.

Uh,

> most economically viable activities better than humans can.

If consider humans as of, say, 1800, then it looks like that goal was achieved long ago via cars, trucks, an electric circle saw, several of the John Deere products, electric lights, synthetic fabrics, nearly all of modern medicine (so far saved my life 4 times), cotton pickers and the rest of cotton processing, canned foods, nearly everything we will have at Thanksgiving dinner this year (apple pie, pecan pie, shrimp), ....

> for better than

For today, look at some John Deere videos!!! They have some big machine that for a corn field does the harvesting while the operator can mostly just watch, monitor, type email to his sweetheart, listen to Taylor Swift. As I recall, the machine even uses GPS to automate the steering!

That is far "better than" what my father in law did to harvest his corn!

So,

> most economically viable activities

is like a moving goal (goal post). Uh, humans are still plenty busy, e.g., writing good software, doing good research, Taylor Swift (supposedly worth $750 million) before her present world tour. Uh, I REALLY like Mirella Freni:

https://www.youtube.com/watch?v=OkHGUaB1Bs8

Sooo, defining the goal is a bit delicate: Need to be careful about nearly, what activities, and when?

Nearly all activities? Sort of already done that. What nearly all people do? Tough goal if the humans keep finding things to do AGI can't yet. I.e., we can keep giving the grunt work to the AGI -- and there is a lot of grunt work -- and then keep busy with what the AGI can't do yet in which case the nearly is a moving goal.

AGI, hurry up; there's a lot to do. For a start, I have some plans for a nice house, and the human workers want a lot of money. My car could use an oil change, and the labor will cost a lot more than the oil -- and I would have to tell the mechanic to be sure to trigger the switch that says there was just an oil change so that the car will know when to tell me it is time for another.

Yes, my little geometry problem with my thinking does qualify as a test of intelligence but due to the nearly and how delicate the definition is can fail the OpenAI test.

I don't see the current OpenAI work, the current direction of their work, or their definition of AGI as solving the geometry problem.

There is an activity my startup is to do: Some billions of people do this activity now. My startup should do the activity a lot better than the people or any current solution so should be "economically viable". I doubt that OpenAI is on track to do this activity nearly as well as my startup -- more, say, than the geometry problem. And I do not call my startup AGI or AI.

This situation stands to be so general that the OpenAI goal of nearly all will likely not be the way these activities get automated. Maybe 20 years from now when "nearly all" the activities are quite new and different, maybe the work of OpenAI will have a chance.

We want to remove the human from the solution design. It would help us tremendously tbh, just like I don't know, Google map helped me never to have to look for direction ever again ?