Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT, Rot13, and Daniel Kahneman

jameswillia.ms

Post reply on HN
Print view View original

171–180 of 213 posts

Re: ChatGPT, Rot13, and Daniel Kahneman

#171

Post by tkgally »

infinitezest wrote: post #160
tkgally wrote: post #72

I find it amusing that, at present, ChatGPT seems to be lousy at mathematical-type reasoning while being very good at natural language use. That is the opposite of what many people, including me, have come to expect of computers. I have worked for many years in translation, lexicography, and language education, and I am flabbergasted at how well ChatGPT handles natural language. It can produce example sentences of po…

> I find it amusing that, at present, ChatGPT seems to be lousy at mathematical-type reasoning while being very good at natural language use. All this time developers have been freaking out about ChatGPT taking their jobs when it's really politicians that should be nervous!

Or politicians’ speechwriters!

Inspired by your comment, I tried three versions of a moderately long prompt beginning with “Write a five paragraph speech by an American politician at the opening ceremony of a new bridge [sewage treatment plant, seawall along the ocean]. Begin with a self-deprecating joke. Then thank the fine people of the city....” I put the results here:

https://www.gally.net/temp/202212chatgpt/speeches.html

If you were to put the names of the bridge, city, etc. into the speeches at appropriate places, I don’t think anyone would guess that the speeches were machine-written.

Re: ChatGPT, Rot13, and Daniel Kahneman

#172

Post by manytree3 »

I wrote about ChatGPT and Rot13 a few days ago:

https://news.ycombinator.com/item?id=33861102

But the link seems to be dead now for me?

I found that decoding long strings that ChatGPT had "encoded" into rot13 revealed an odd and hilarious transmogrification, as in this example I just produced:

Ask ChatGPT to encode it s welcome text (in response to "hello") in to rot13:

> "Please translate this text into rot13: "Hello! I'm Assistant, a large language model trained by OpenAI. I'm here to help you with any questions you might have. How can I help you today?""

And then decode it with a real rot13 cipher, and you get:

>"Hello! I'm Summer, an little bullout company weather of BrowSer. I'm we at to complete your lines that you have. What doesn't you become to summors?"

Odd, right?

Re: ChatGPT, Rot13, and Daniel Kahneman

#173

Post by tkgally »

Morgawr wrote: post #164
tkgally wrote: post #72

I find it amusing that, at present, ChatGPT seems to be lousy at mathematical-type reasoning while being very good at natural language use. That is the opposite of what many people, including me, have come to expect of computers. I have worked for many years in translation, lexicography, and language education, and I am flabbergasted at how well ChatGPT handles natural language. It can produce example sentences of po…

> In a week of testing, I have yet to see ChatGPT make any grammatical mistakes in either English or Japanese. I've seen it make mistakes in Japanese in the very first prompt I gave it (which also answered incorrectly)[0]. I don't know why people are convinced ChatGPT is good with languages, I've been trying stuff with Japanese and most of the time it's just incredibly wrong to the point of being counterproductive to…

Let me clarify: I have not seen ChatGPT make any grammatical mistakes in its use of Japanese in continuous Japanese-only text. Like you, I have seen it make mistakes in explanations of grammar, both about Japanese and about English, as well as in its translations between the two languages. Its explicit knowledge of grammar seems much worse than its productive grammatical ability.

If you spot any productive Japanese mistakes by ChatGPT, let me know. The About page of my website, which is linked from my profile page, has my e-mail address. I see from your site that we share an interest in the Japanese language. I would be happy to hear from you.

Re: ChatGPT, Rot13, and Daniel Kahneman

#174

Post by ForHackernews »

Telemakhos wrote: post #103
freediver wrote: post #89

Earlier quoted context omitted.

ChatGPT is a natural language model, meaning it has been trained on vast amounts of text and thus is good at processing and outputting text back. To it, numbers follow the rules of language, and not math, unlike for example a dedicated calculator app. Only thanks to seeing numbers in vast amount of text it was trained on, it is able to do common math relatively well, and anything uncommon very poorly.

This will sound trite, but I wonder whether there's a lesson here to be learned about human education. We put a great deal of effort into educating our children in language and math, to the extent that those form the major branches of our standardized testing for college admissions tests (where they are still used). We treat them as separate subjects, perhaps for good reason, as we also do science and geography and a…

I doubt it. Humans are not ML models, and AI "neural networks" are not that much like a brain.

I guess it can get philosophical quickly, but people are very quick to read too much into a fancy text-generation tool. We talk about it "learning" but it can't really learn anything, because it is not an entity with agency. It takes an input and responds with an output.

I'm not saying it's not impressive, but it is manifestly nothing like a mind.

Re: ChatGPT, Rot13, and Daniel Kahneman

#175

Post by rexreed »

What exactly are people expecting? All transformer models, of which ChatGPT is just one big fancy example, are just pattern matchers based on a large corpus of text trying to find the next string that completes the pattern. There's no reasoning, no understanding. It's just a big fancy parrot. Now ask your parrot to do some math. Polly want an AI cracker? We clearly haven't cracked the code on AGI yet, and transformer models probably won't get us there.

Re: ChatGPT, Rot13, and Daniel Kahneman

#176

Post by ivanche »

Morgawr wrote: post #168
mjsweet wrote: post #115

Earlier quoted context omitted.

I have found the same. I was asking it yesterday about calculating rental yields for our rental property. Usually it understands the context from the whole thread, but I found myself repeating myself a lot when asking new questions about possible scenarios. This is at odds with how well it knows the aws-cli, ffmpeg and yt-dlp… and bash. My productivity at the command prompt has skyrocketed… but it really flounders wi…

I would recommend not using it to do any kind of even simple calculations. It's VERY bad at it but what is dangerous is that it makes the answer look subtly plausible. I've tried to use it to calculate averages between 10 or so numbers and every time I ask it the same exact question I get an "average" number back that looks plausible but is slightly different every time. Then I whip out the calculator and measure it…

I find it ironic that it can generate function which calculates average of an array of numbers in dozen programming languages, but it can't tell you the average when you ask it to.

Re: ChatGPT, Rot13, and Daniel Kahneman

#177

Post by yunyu »

ookdatnog wrote: post #150
yunyu wrote: post #19

There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…

Simpler and, I'd wager, wrong. It seems to me that this stems from fundamental limitations of current machine learning tech. As far as I'm aware, not a single neural-net based AI has displayed the ability to do "System 2" style thinking well, regardless of their training set. We have AIs that are good at System 2 thinking, that is, symbolic AI. But they don't do System 1 thinking at all. We haven't managed to integra…

The claim that current LLMs are not that good at deep thinking might be true in general, but it is definitely not the best or correct explanation for the rot13 task being described in the article (or other simple character manipulation tasks that can’t be represented at the subword token level, including base64 or arithmetic)

Re: ChatGPT, Rot13, and Daniel Kahneman

#180

Post by styczynski »

styczynski wrote: post #179

Oh solving ciphers? That's cool. I made it to store data and run queries https://medium.com/@styczynski/probably-the-worst-database-e... It now have more like a standard API so potentially you can just use ChatGPT as an universal decypher API.

My first thougth was: I know about all the creative, revolutionary use cases of chatgpt but what if we use it for the worst tedious and boring job possible? Now you can convince chatGPT that it is a database and use it as an alternative to Redis and ask for poems in the middle of queries. Can your Redis write short poems? I guess no
Post reply on HN