Wolfram Alpha and ChatGPT

You're right. How could I know this? I have formed an opinion about a person I have never personally met. Mea culpa. My impression then is that Stephan Wolfram, in spite of his considerable and quite & justifiably impressive brain, refuses to apply the necessary corrective measures to adjust for the fact of the existence of external agencies in the world when formulating his personal theory of the world.

> a bot that is almost always wrong

"It’s always amazing when things suddenly “just work”. It happened to us with Wolfram|Alpha back in 2009. It happened with our Physics Project in 2020. And it’s happening now with OpenAI’s ChatGPT. "

It is possible I missed the widespread excitement about Wolfram|Alpha and Project Physics. The former did make waves in geek circles, I remember that. The latter did not make it to the New York Times, did it?

https://www.google.com/search?q=Wolfram%7Calpha+stephan+wolf...

https://www.google.com/search?q=Project%20Physics%20stephan%...

https://www.google.com/search?q=ChatGPT

Coping with that, in his specific case.

When I ask it the same questions my results are different:

Q: What Is the distance from Chicago to Tokyo? A: The distance from Chicago, United States to Tokyo, Japan is approximately 8,113 miles (13,055 kilometers) if you were to fly in a straight line. It takes approximately 15 hours of flight to reach from Chicago to Tokyo.

Which is fun because it is also wrong, but gives a different incorrect value.

Q: What is 3 to the power 73?

A: 3 to the power 73 is equal to 3 raised to the 73rd power, which can be calculated by multiplying 3 by itself 73 times. 3^73 = 3 * 3 * 3 * ... (73 times) = 3 * 3 * ... * 3 (73 times) = 1.43382043e+22 So 3 to the power 73 is approximately 1.43382043e+22

Also incorrect but a different value. If I change the question slightly, for example, by dropping a question mark or saying "Three to the power OF 73" It also gives a slightly different incorrect answer.

What I find really interesting here is that if I ask it the exact same question it gives the exact same result. But then if I open up a new instance, it will give a different incorrect answer, and repeat the incorrect answer again only if the question is identical.

Edit: This could be a decent method of fact checking for anyone determined to use chatGPT; phrase the question slightly differently and compare the results (never input the exact same question twice).

Interestingly, it now correctly outputs Honduras for the second largest country in South America, but if you ask it to list them by size it will get most of the rest incorrect.

My own experimentation with ChatGPT made me dismiss it, but I was asking it comparatively difficult questions about linear algebra and programming. I'm kind of shocked it fails at these basic questions I would have thought it would be more than capable of handling.

I'm with you there.

I started using Mathematica 22 years ago, during first year of college. I've worked as a professional Python developer for slightly less time (18 years). I've built tons of systems in Python, and none in Mathematica, yet Mathematica remains as the best option as a "bicycle for the mind", and that's thanks to the coherent vision of Wolfram research and perhaps Wolfram himself.

One thing to add is that Wolfram, as you said, has money. That means he doesn't spend his days fixing bugs in a devops pipeline and questioning his life choices while a scrum master impishly consumes his days on Earth with inanity. He has time to think about stuff, to explore ideas, and he has already explored quite a bit, and has a breath of understanding of math, physics and a few other sciences that the average machine learning engineer simply doesn't have.

So, yeah, I'm a Python fanboy, but honor where honor is due.

“ Finally, and suddenly, here’s a system that can successfully generate text about almost anything—that’s very comparable to what humans might write. It’s impressive, and useful. And, as I’ll discuss elsewhere, I think its success is probably telling us some very fundamental things about the nature of human thinking.”

I think that being able to analyze, preprocess and remix a huge corpus of human-authored text on all subjects is altogether a different type of “intelligence” than actually understanding the subject.

In some cases, it can approach understanding and actually demonstrate it. In others, it gets things woefully wrong - such as when it claims bears are larger than elephants and shows figures to back it up that prove the exact opposite. (I asked if a bear could eat an elephant.)

As I started to ask ChatGPT the same question with different parameters, I saw the Mad Libs side of it. (Simply replacing text in various positions.) It has a TON of that.

Also I don’t know how much its many, many human handlers massaged the basic structures, eg “why is this joke funny” etc. That kind of stuff is the type of Artificial Intelligence that all science and technology is - many hands work on software and we ship the next version. That is itself an intelligent process.

HOWEVER, what I am really looking forward to is “chain of reasoning” advances. Can anyone point me to those?

Also, has Cyc gone anywhere? Wolfram should be aware of that project.

Bad assumption. I just logged into a very separate Google account and did the same query. Result:

6,298 mi Distance from Chicago to Tokyo