Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wolfram Alpha and ChatGPT

writings.stephenwolfram.com

Post reply on HN
Print view View original

231–240 of 309 posts

Re: Wolfram Alpha and ChatGPT

#231

Post by ben_w »

machina_ex_deus wrote: post #187
ben_w wrote: post #98

Earlier quoted context omitted.

Which of these hypotheticals is least bad : an AI which won't write political invective against anyone, or one which will be used by your enemies to stir up hatred against your entire team, and your team's only available response is to do the same back at the entire other side?

How about looking at political conversation as less of a fight and more of a dialogue? In which case I wouldn't mind any intelligent input from chatGPT even if it's against my viewpoint, and nobody really needs artificial stupidity as there's already plenty of human stupidity to go around. Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so call…

> Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"")

Most humans can’t help but be tribalist, which is why I don’t blame you for taking a swipe at social justice even though I very carefully didn’t say which tribe I’m in.

Now, given you appear to hate social justice enough to put it in scare quotes, I ask you to imagine some amoral but intellectually perfect AI tasked with the goal of creating propaganda that calls for the destruction of anyone who hates social justice. (If you’d said the same thing but put “conservatives” in scare quotes, I’d make the same point but with the phrase “anyone who hates conservatism”).

Telling me not to be tribal isn’t going to stop someone sending that prompt to that AI. Telling the AI not to be, is going to stop them getting infinite free propaganda.

This isn’t the only reason to do this (the massive PR risk from an unfettered AI is another), but it is a sufficient reason to do this, and OpenAI have repeatedly demonstrated that they err on the side of assuming their AI is better than it really is just in case.

Re: Wolfram Alpha and ChatGPT

#232

Post by countvonbalzac »

pdonis wrote: post #190
agentwiggles wrote: post #18

I'm almost offended by the "cubic light year of ice cream" answer from ChatGPT. It's obviously ridiculous but is also a fairly simply dimensional analysis problem. Do the damn math, don't wag your finger at me and crush my dreams! I'm pretty bullish on ChatGPT and its ilk, but I _really_ dislike when ChatGPT lectures me because my request is against its "moral values." I recently pasted in the lyrics from Sleep's tit…

> Do the damn math Wolfram's point, which is valid, is that ChatGPT can't do the damn math. That's simply not what it does. To do things like do accurate math, you need a different kind of model, one that is based on having actual facts about the world, generated by a process that is semantically linked to the world. For example, Wolfram uses the example of asking ChatGPT the distance from Chicago to Tokyo; it gives…

Glad to see others recognize Wolfram's assertion that he is god's gift to the field of computer science

Re: Wolfram Alpha and ChatGPT

#233

Post by sebzim4500 »

dragonwriter wrote: post #192
pdonis wrote: post #190

Earlier quoted context omitted.

> Do the damn math Wolfram's point, which is valid, is that ChatGPT can't do the damn math. That's simply not what it does. To do things like do accurate math, you need a different kind of model, one that is based on having actual facts about the world, generated by a process that is semantically linked to the world. For example, Wolfram uses the example of asking ChatGPT the distance from Chicago to Tokyo; it gives…

> To do things like do accurate math, you need a different kind of model, one that is based on having actual facts about the world, generated by a process that is semantically linked to the world. Or you just need a model that can recognize math, and then pass it to a system that can do math. Math is actually something traditional, non-AI systems are very good at doing (it is the raison d’être of traditional computin…

Right, that's basically the suggestion made in the article.

Re: Wolfram Alpha and ChatGPT

#234

Post by thomastjeffery »

Winsaucerer wrote: post #210
thomastjeffery wrote: post #202

The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…

> As I see it, there is clearly no way to advance ChatGPT into anything more than it is today. Impressive as it is, the curtain is wide open for all to see, and the art can be viewed plainly as what it truly is: magic, and nothing more. (I only RTFA after writing this comment, and I now see that the below is what they're doing) I'm an outsider to this field. My unexpert thought was that perhaps this model could be us…

You would be trading the problem for another instance of the same problem.

When you ask ChatGPT to construct a mathematical question, it will do so the same way it does everything else: by semantic popularity.

And that is the problem we are trying to avoid. The semantically popular guess might be logically sound, but it might not. It's a gamble no matter when or where it is done.

--

All it takes is what I call a "semantic off-by-one error". That might look like our first problem:

> Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result?

The problem is that a semantically close response is nothing like "234241.24211 + 58352342.52544". It's just going to be whatever arbitrary text that already exists in the training dataset is closest to the semantic phrasing of the question. That might be the correct number, is more likely to be an incorrect number, and is even likely to be a wordy response.

--

So if we follow your thought process to interrupt the guesswork, it would involve restructuring the prompt.

> Please restructure the following prompt into a mathematical statement: "Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result?"

The output you are hoping for

>> 234241.24211 + 58352342.52544

Another completely valid and possible output:

>> 234241.24211 / 58352342.52544

Another:

>> 2424332.34434 + 535858932.5358

--

There is no way to guarantee the reformulated question is logically equivalent to its original. That's the problem, and the problem cannot be moved. With every step, ChatGPT is guessing. ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is.

The only place you can pause an interaction with ChatGPT to do some logic is instead.

Re: Wolfram Alpha and ChatGPT

#235

Post by thomastjeffery »

CrushItNerds519 wrote: post #225
thomastjeffery wrote: post #202

The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…

This comment and many others speculate on the limits of ChatGPT based on assumptions about what ChatGPT does that are not quite accurate. In particular, ChatGPT does not simply output the “most semantically popular result”. That description applies only to the base model, before instruction tuning and RLHF. As for the speculation itself, e.g., “as soon as you merge two subjects, you are right back to gambling semanti…

Much like ChatGPT, you seem to have comprehended the words I said, but not their meaning.

Re: Wolfram Alpha and ChatGPT

#236

Post by ben_w »

int_19h wrote: post #216
ben_w wrote: post #98

Earlier quoted context omitted.

Which of these hypotheticals is least bad : an AI which won't write political invective against anyone, or one which will be used by your enemies to stir up hatred against your entire team, and your team's only available response is to do the same back at the entire other side?

In practice, it ends up being an AI that won't do the former for your average person but can still be prompt-engineered to do the latter by a sufficiently determined attacker.

Probably. But AI alignment research is currently extremely primitive (I think they describe themselves as “pre-paradigmatic”), so giving them some time to find a better way is at least worth trying.

Re: Wolfram Alpha and ChatGPT

#237

Post by thomastjeffery »

cj wrote: post #228
thomastjeffery wrote: post #202

The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…

This is the best description of the state of ChatGPT that I’ve read thus far. …Unless this whole comment was generated by ChatGPT. (I strongly dislike that I’m starting to second guess whether comments are written by humans or not)

That's because everything written by ChatGPT is a transformation of other stuff that was written by humans.

If we trained it on gibberish and nonsense, we would get that, and no one would care. The tricky part is that we don't have much gibberish or nonsense to train with: because of that pesky property of human expression we call logic.

Re: Wolfram Alpha and ChatGPT

#238

Post by SamPatt »

asoneth wrote: post #217
TylerLives wrote: post #144

Earlier quoted context omitted.

Surely this can be done better without ChatGPT? One thing I can think of is doing it on internet forums. Somebody could use lots of accounts to generate content like that on HN. Now that I think about it, this seems unavoidable and I don't see how public forums can defend against it.

I suspect that the goal of the OpenAI/GPT usage restriction is not to prevent Every Bad Thing, but to avoid contributing to them in a way that draws negative attention. After all, "Political Campaign Uses AI to Make Attack Ad" would get more clicks than "Political Campaign Makes Attack Ad" so there will be extra journalistic scrutiny whenever AI is involved.

Same with their attempt at watermarking the AI output.

Re: Wolfram Alpha and ChatGPT

#239

Post by theptip »

janalsncm wrote: post #224
theptip wrote: post #129

Earlier quoted context omitted.

I don't think most of the interesting knowledge encoded in Wolfram Alpha changes. Mathematics and pure Logic is true, and immutable. Most of Physics, ditto.

Thats true for the physics and math but some of the things are updated all the time. For example you can get a current weather report. And they have structured data about movies, TV shows, music, and notable people [1]. Every time any country has an election you’re going to retrain your model? That gets really expensive really fast. On top of that, the training process isn’t that trustworthy. There’s no guarantee you…

Right, I see what you're getting at. I do agree that AI systems will need to be able to use oracles, "current weather" is a great example of something a human also looks up.

The reason I want the model itself to learn Physics, Maths, etc. is that I think it is going to end up being critical to the challenge of actually developing logical reasoning on a par or above humans, and to gain a true "embodied understanding" of the real world.

But yeah, it would be nice to have your architecture support updating facts without full retraining. One approach is to use an oracle as you note. Another would be to have systems do some sort of online learning (as humans do). (Why not both?) The advantage of the latter approach is that it allows an agent to deeply update their model of the world in response to new facts, as humans can sometimes do. Anything that I'm just pulling statelessly from an oracle cannot update the rest of my "mind". But this is perhaps a bit speculative; I agree in the short-term at least we'll see better performance with hybrid LLM + Oracle models. (As I noted, LaMDA already does this.)

However I think that a big part of Wolfram's argument in the OP is that he thinks that an LLM can't learn Physics or Maths, or reliably learn static facts that a smart human might have memorized like distances between cities. And that's the position I was really trying to argue against. I think more scale and more data likely gets us way further than Wolfram wants to give credit for.

Re: Wolfram Alpha and ChatGPT

#240

Post by [deleted] »

CJefferson wrote: post #174

One general comment I'll give to this. Combining neural networks (like ChatGPT) and logical (like Wolfram Alpha) AI systems has been the aim of many people for 30 years. If someone manages it well, it will be a massive step forward for AI, probably bigger than the progress made by the GPTs so far. However, while there are lots of ideas, no-one knows how to do it (that I know of), and unlike the GPTs, it isn't a probl…

[deleted]
Post reply on HN