I Taught ChatGPT to Invent a Language • Hacker News Board

Re: I Taught ChatGPT to Invent a Language

#171

Post by Oxidation » Tue, Dec 06, 2022, 10:51 PM UTC

mwigdahl wrote: post #154

Miraste wrote: post #134

Earlier quoted context omitted.

It insisted to me for a while that the sun was red. The "guardrails" do nothing to stop it from lying confidently about almost everything.

A lot of people say that the model is "lying" when it returns incorrect facts. Generally when friends or family confidently assert facts that they are sure but also incorrect about, we call them "misinformed" rather than connoting malice with the word "lying". Have they heard the right facts but associated them incorrectly? Heard the wrong facts from whatever source they obtained them from? Either or both could also…

It's more like you can't actually read but you've seen a lot of media about cars that contain text saying they have a thing called a "width" around the 1.7-2m mark and usually aren't round numbers. So when someone asks you how wide a 2011[1] Ford Focus is[2], you say "1,769mm" and look confident. Syntactically right and by any probabilistic model, pretty good and plausible.

[1] and if you ask for 2012, it tells you "I'm sorry, but I am not able to browse the internet to provide you with the specific dimensions of the 2012 Ford Focus."

[2] according to Google results, it's 1823 mm to 1842 mm.

Re: I Taught ChatGPT to Invent a Language

#172

Post by layer8 » Tue, Dec 06, 2022, 10:52 PM UTC

assusdan wrote: post #11

datalopers wrote: post #4

Earlier quoted context omitted.

ChatGPT access is 100% free and instant approval.

Yet it is not available in some countries. VPN does not help, as valid phone number is required and its country code is being checked.

You could try something like https://receive-smss.com/.

Re: I Taught ChatGPT to Invent a Language

#173

Post by lgessler » Tue, Dec 06, 2022, 10:55 PM UTC

Speaking as someone who works in AI and as a linguist: this is really impressive, but be aware the grammatical phenomena involved here are common enough that they were surely well represented in linguistic texts in GPT3.5's training corpus. I'd be more curious to see how it would perform if you gave ChatGPT high-level instruction on more exotic grammatical phenomena, such as sandhi, split-ergativity, and Austronesian alignment.

Re: I Taught ChatGPT to Invent a Language

#174

Post by skytrue » Tue, Dec 06, 2022, 10:55 PM UTC

allturtles wrote: post #32

bjourne wrote: post #3

Someone with ChatGPT access able to replicate? Because this article is unbelievable.

I repeated the same prompts for the first 10 or so prompts and got similar but less consistent results. Here's the last prompt I repeated: > Excellent. How would I say "The slime eats food with its mouth"? In Glorp, the sentence "The slime eats food with its mouth" could be translated as "Slimb snarf sploota slimbe," using the vocabulary and grammar that we have defined so far. Here is how the words in the sentence c…

I feel like people are forgetting that these models are non-deterministic. You can give it the exact same input and get a different response, so "testing" in this way doesn't work. That's also why (right now) automation is impossible. You aren't guaranteed an expected output.

Re: I Taught ChatGPT to Invent a Language

#175

Post by WaxProlix » Tue, Dec 06, 2022, 10:59 PM UTC

lgessler wrote: post #173

Speaking as someone who works in AI and as a linguist: this is really impressive, but be aware the grammatical phenomena involved here are common enough that they were surely well represented in linguistic texts in GPT3.5's training corpus. I'd be more curious to see how it would perform if you gave ChatGPT high-level instruction on more exotic grammatical phenomena, such as sandhi, split-ergativity, and Austronesian…

That's not really less impressive; if this pattern matcher did as good a job at some sort of acquisition/generation task (no poverty of stimulus argument here...) and figured out the rules in a productive fashion, that's still really cool.

Strong universality in a super-Chomskyan manner (ie, all universal features are available from the acquisition of A language) would be... neat, too, but I wouldn't expect it.

Re: I Taught ChatGPT to Invent a Language

#176

Post by [deleted] » Tue, Dec 06, 2022, 11:02 PM UTC

fnordpiglet wrote: post #168

furyofantares wrote: post #144

Earlier quoted context omitted.

I agree with you, and it's amazing. But the claim was that it's an example of how much more you can get done with ChatGPT when it seems like an example where the author got a lot less done than they would have with notepad.exe

But this isn’t an example of that it’s an example of “wow that’s amazing” In the realm of getting a lot more stuff done, I’ve been using it as a companion to explain a new language I’m learning right now. It’s super useful to ask it about language constructs, idiomatic ways to do X, is this way of doing Y good, etc. It’s saved me hours of using Kagi to soft through semi relevant pages stuffed with SEO and ancient ans…

[deleted]

Re: I Taught ChatGPT to Invent a Language

#177

Post by cecilpl2 » Tue, Dec 06, 2022, 11:02 PM UTC

jameshart wrote: post #149

I would definitely question who is doing the inventing here. However, I had an experience with chatGPT yesterday that definitely felt like it actually inventing. Problem is, I didn’t ask it to. I was seeing how much it knew about HyperTalk, the scripting language used in HyperCard. And it was surprisingly well informed. But it also told me that one of the interesting things about HyperTalk was its ‘message stack’ sys…

I tried several times to play chess against it. It made reasonable if poor moves for the first 10 moves of the game. It also gave plausible-sounding if entirely incorrect explanations of why the moves were good, such as "I'm playing Nf6-g8, which is a good move because it defends my pawn on f7 [it doesn't] and controls the center, allowing me to set up an attack on your king".

At some point though, it always got stuck and insisted on making illegal moves, even when I corrected it by providing the current board position in FEN.

Re: I Taught ChatGPT to Invent a Language

#178

Post by furyofantares » Tue, Dec 06, 2022, 11:05 PM UTC

fnordpiglet wrote: post #168

furyofantares wrote: post #144

Earlier quoted context omitted.

I agree with you, and it's amazing. But the claim was that it's an example of how much more you can get done with ChatGPT when it seems like an example where the author got a lot less done than they would have with notepad.exe

But this isn’t an example of that it’s an example of “wow that’s amazing” In the realm of getting a lot more stuff done, I’ve been using it as a companion to explain a new language I’m learning right now. It’s super useful to ask it about language constructs, idiomatic ways to do X, is this way of doing Y good, etc. It’s saved me hours of using Kagi to soft through semi relevant pages stuffed with SEO and ancient ans…

Your post seemed dismissive of mine, given the tone of suggesting I just try to get my laptop to generate a slime language from the command prompt or something. But my post was a reply to "This is a great example of how much more you can get done with tools like ChatGPT" and it seems like you agree that it isn't.

Re: I Taught ChatGPT to Invent a Language

#179

Post by lgas » Tue, Dec 06, 2022, 11:07 PM UTC

jrmg wrote: post #165

jameshart wrote: post #149

I would definitely question who is doing the inventing here. However, I had an experience with chatGPT yesterday that definitely felt like it actually inventing. Problem is, I didn’t ask it to. I was seeing how much it knew about HyperTalk, the scripting language used in HyperCard. And it was surprisingly well informed. But it also told me that one of the interesting things about HyperTalk was its ‘message stack’ sys…

I asked how to configure something in PlatformIO and it told me it was possible to do exactly what I wanted, and gave me easy to use syntax with full examples and explanation. When I questioned it further, it pointed me to the URLs of docs covering the feature. None of it was in any way real - the feature I wanted does not actually exist.

> the feature I wanted does not actually exist.

Did you try asking it to implement it?

Re: I Taught ChatGPT to Invent a Language

#180

Post by Oxidation » Tue, Dec 06, 2022, 11:08 PM UTC

fnordpiglet wrote: post #169

Miraste wrote: post #134

Earlier quoted context omitted.

It insisted to me for a while that the sun was red. The "guardrails" do nothing to stop it from lying confidently about almost everything.

I’ve only seen it do stuff like that if I assert a false premise and ask it questions off the false premise.

It seems also to do it if you ask about something that's not documented in detail. It'll get details of the 1976 Tangshan earthquake dead on, but ask for the 1982 Bonin Island earthquake and apparently it was at a depth of 40km (it was actually 22.2), but it gets the magnitude right.

The Nicobar Islands earthquake on 26 December the same year apparently killed 4000 with a magnitude of 7.1. It didn't, it injured 52, and there were two: magnitude 6.2 and 6.3. And it was in January. 26th December was indeed an earthquake that affected those islands. In 2004.

Which is not that surprising, as how would it know, considering it probably didn't have much input about it. What is more amazing is the seamless blending of correctly regurgitated details with very plausible wholesale fabrication and, even more convincingly, blending of fact with fiction.