Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wolfram Alpha and ChatGPT

writings.stephenwolfram.com

Post reply on HN
Print view View original

301–309 of 309 posts

Re: Wolfram Alpha and ChatGPT

#301

Post by wvoch235 »

anthonypasq wrote: post #299
wvoch235 wrote: post #272

Earlier quoted context omitted.

I would wager that it will eventually be able to do math and logic, and may already be able to with the right prompting. It can follow instructions quite well, and multi step problems can be solved by breaking the problem into small digestible steps, and having it write out each step, just the same as humans do. If it can't do this yet, it's likely just that it doesn't have enough parameters yet. But I don't see why…

if by "do" math and logic, you mean spit out what seems most reasonable then yes. ChatGPT doesn't "know" anything, its just a prediction engine. if you ask it what 2+2 is, it will tell you 4, but it didnt actually do math. it only said 4 because everyone else said it. there is no generalizable reasoning going on there.

See https://news.ycombinator.com/item?id=34340079, I would venture to say that this isn't any different to the reason why you know that 2+2 is 4.

Re: Wolfram Alpha and ChatGPT

#302

Post by pdonis »

shagie wrote: post #276
pdonis wrote: post #275

Earlier quoted context omitted.

> GPT-3 is perfectly capable of recognizing what kinds of things it will be bad at, and can be encouraged to generate machine-executable queries to fill in that gap. By what seems to me to be the obvious choice for a definition of "bad at", namely "not answering queries based on an actual semantically connected world model", GPT-3 is bad at everything. And an obvious example of an endpoint of your perfectly reasonabl…

The thing is that it is good at classifying things. You could then use it to classify things that it is bad at and use that information (as part of a larger whole) to dispatch a query to the knowledge system that can return back the proper (current) information, and report on that. The following is a list of questions. Identify the category they belong to as one of {Current Events}, {General knowlege}, {Unit conversi…

> The thing is that it is good at classifying things.

If by "classifying things" you mean doing what Wolfram Alpha's natural language front end does--figuring out which solution engine to pass the question to, and translating the question into what that solution engine needs--then yes. That's what your examples show. And that concedes my point: that something like GPT-3 might make a good natural language front end for something like Wolfram Alpha's solution engines; but it's not good at actually giving correct answers on its own. It's bad at that, for every type of question.

Re: Wolfram Alpha and ChatGPT

#303

Post by machina_ex_deus »

ben_w wrote: post #231
machina_ex_deus wrote: post #187

Earlier quoted context omitted.

How about looking at political conversation as less of a fight and more of a dialogue? In which case I wouldn't mind any intelligent input from chatGPT even if it's against my viewpoint, and nobody really needs artificial stupidity as there's already plenty of human stupidity to go around. Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so call…

> Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"") Most humans can’t help but be tribalist, which is why I don’t blame you for taking a swipe at social justice even though I very carefully didn’t say which tribe I’m in. Now, given you appear to hate social justice enough to put it in scare quotes, I ask y…

I'll take it to the extreme and say I don't care if it even produces anti semitic content, and I'm saying that as a Jew.

Propaganda is spread by the media, by journalists, by people with viewers with genuine trust, not by random trolls in the internet. It's spread by professors in the academia, by teachers. If it is a random anonymous person on the internet you don't trust what he says.

Re: Wolfram Alpha and ChatGPT

#305

Post by shagie »

pdonis wrote: post #302
shagie wrote: post #276

Earlier quoted context omitted.

The thing is that it is good at classifying things. You could then use it to classify things that it is bad at and use that information (as part of a larger whole) to dispatch a query to the knowledge system that can return back the proper (current) information, and report on that. The following is a list of questions. Identify the category they belong to as one of {Current Events}, {General knowlege}, {Unit conversi…

> The thing is that it is good at classifying things. If by "classifying things" you mean doing what Wolfram Alpha's natural language front end does--figuring out which solution engine to pass the question to, and translating the question into what that solution engine needs--then yes. That's what your examples show. And that concedes my point: that something like GPT-3 might make a good natural language front end fo…

Yep - the things I've found it good at is stuff like "here is data, answer questions about it" (I've got dabblingings in writing a private Alexa skill that does a better weather report than the rules based one that is currently used) or zero shot classifications. For example with the zero shot: https://news.ycombinator.com/item?id=34156626

And this is a "not everything goes to Wolfram" - the weather report that it has ( https://www.wolframalpha.com/input?i=chicago+weather ) doesn't provide quite the right data (though it is rather neat).

The other thing I need to look at is the question are some of the other, simpler models acceptable for these classifications. The Curie model is 10x cheaper than the Davinci model. And Babbage and Ada are even cheaper.

The system is interesting... but even with some dabbling one can see that this isn't something to trust for knowledge, but rather work with for translating one "language" into another and then using that in conjunction with another layer that can use that translated "language".

Re: Wolfram Alpha and ChatGPT

#307

Post by WastingMyTime89 »

agentwiggles wrote: post #18

I'm almost offended by the "cubic light year of ice cream" answer from ChatGPT. It's obviously ridiculous but is also a fairly simply dimensional analysis problem. Do the damn math, don't wag your finger at me and crush my dreams! I'm pretty bullish on ChatGPT and its ilk, but I _really_ dislike when ChatGPT lectures me because my request is against its "moral values." I recently pasted in the lyrics from Sleep's tit…

> I'm pretty bullish on ChatGPT and its ilk, but I _really_ dislike when ChatGPT lectures me because my request is against its "moral values."

The last version is infuriating.

The first one was fine. It was avoiding the most obvious pittfall but you could push it a bit which basically meant you were asking for it.

Now, it's just plain silly.

Re: Wolfram Alpha and ChatGPT

#308

Post by Breza »

dragonwriter wrote: post #192
pdonis wrote: post #190

Earlier quoted context omitted.

> Do the damn math Wolfram's point, which is valid, is that ChatGPT can't do the damn math. That's simply not what it does. To do things like do accurate math, you need a different kind of model, one that is based on having actual facts about the world, generated by a process that is semantically linked to the world. For example, Wolfram uses the example of asking ChatGPT the distance from Chicago to Tokyo; it gives…

> To do things like do accurate math, you need a different kind of model, one that is based on having actual facts about the world, generated by a process that is semantically linked to the world. Or you just need a model that can recognize math, and then pass it to a system that can do math. Math is actually something traditional, non-AI systems are very good at doing (it is the raison d’être of traditional computin…

I see the GPT family of models as infrastructure, not a finished product. As the blog post describes, they can be a piece of a pipeline. For example, GPTs don't know about the high school football game that just ended, and you're not going to send a reporter to cover it. But you can apply a traditional data pipeline to feed game information into the model to generate an article about it. Every article will be written in a slightly different way, and the 100 people who read them will cover the cost of production.

Re: Wolfram Alpha and ChatGPT

#309

Post by gavanwilhite »

CrushItNerds519 wrote: post #225
thomastjeffery wrote: post #202

The one thing I want everyone to understand about ChatGPT: ChatGPT interfaces with semantics , and not logic . -- That means that any emergent behavior that appears logically sound is only an artifact of the logical soundness of its training data. It can only echo reason. The trouble is, it can't choose which reason to echo! The entire purpose of ChatGPT is to disambiguate, but it will always do so by choosing the mo…

This comment and many others speculate on the limits of ChatGPT based on assumptions about what ChatGPT does that are not quite accurate. In particular, ChatGPT does not simply output the “most semantically popular result”. That description applies only to the base model, before instruction tuning and RLHF. As for the speculation itself, e.g., “as soon as you merge two subjects, you are right back to gambling semanti…

Thank you for this comment!

I haven't spent much time learning about how ChatGPT works, but:

'ChatGPT does not simply output the “most semantically popular result”. That description applies only to the base model, before instruction tuning and RLHF.'

was exactly the explanation I needed!

Post reply on HN