Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Taught ChatGPT to Invent a Language

maximumeffort.substack.com

Post reply on HN
Print view View original

91–100 of 250 posts

Re: I Taught ChatGPT to Invent a Language

#91

Post by CamperBob2 »

dvt wrote: post #37

I know everyone's (as usual) "amazed" by this, but here are a few stark observations: (1) a linguist could do this (come up with a toy language, conjugations, and sentence structure, probably the Python code too) in like 30 minutes, (2) it's disappointing that the AI constantly messes up very simple grammar rules (rules a grade-schooler could understand)—I'm actually very surprised it doesn't have hardcoded grammar r…

"This talking dog makes embarrassing grammatical errors. Besides that, it learned everything it knows from humans, not from other dogs. I don't see the big deal."

Anyone familiar with Chomsky's school of thought here? I'm not sure that "it's just a language model" is the effective dismissal that a lot of people think it is.

Re: I Taught ChatGPT to Invent a Language

#92

Post by Kuinox »

fullstackchris wrote: post #53

I see posts like this and don't understand how they are made. I asked chatGPT this morning if it "remembered" something I asked in the previous prompt, and it said something to the tune of "I am simply a NLP machine learning model, I provide a decent output for good inputs, I don't have a sense of context, memory, or time." That's why I really don't understand these contextualized multi step conversations. Especially…

ChatGPT may have lied to you.

It loves to lie.

You can easily test if it can remember things: https://imgur.com/a/oOvJnln

Re: I Taught ChatGPT to Invent a Language

#93

Post by fnordpiglet »

Oxidation wrote: post #73
fnordpiglet wrote: post #49

Earlier quoted context omitted.

I find it amazing from the point of view that this is not a linguist or a grade schooler, but a general purpose computer program. Have you seen a computer program that can do these things, especially one that wasn’t written to do these things but can also turn itself into a plausible linux box, translate python to rust, and all sorts of other wacky party tricks? That’s the amazing thing - not that the feats themselve…

> some of these gaps can be tackled Seems like the major gap is in facts. It'll often make up completely plausible specific details that are flat wrong, or refuse to proceed if it's "not allowed" to do so. Coupling it (handwave) with a massive structured data repository like Wolfram Alpha and Wikidata would be really something.

I find the guard rails they put in maddening and weirdly implemented. I wanted it to make up jokes for me and it’s jokes are the same few over and over. (I can never remember jokes so this is a super useful tool for me!) so I wanted it to tell me jokes in the style of Jerry Seinfeld. The guard rails prevented it from doing that because he’s a real person and it tries to not bend its personality pretty hard. However by just resetting the convo a few times I randomly got it to agree to tell me a joke. I can’t remember it but it was actually funny and in the style of Seinfeld. shrug

It almost certainly has indexed Wikipedia, fwiw.

Re: I Taught ChatGPT to Invent a Language

#94

Post by robmccoll »

dvt wrote: post #65
fnordpiglet wrote: post #49

Earlier quoted context omitted.

I find it amazing from the point of view that this is not a linguist or a grade schooler, but a general purpose computer program. Have you seen a computer program that can do these things, especially one that wasn’t written to do these things but can also turn itself into a plausible linux box, translate python to rust, and all sorts of other wacky party tricks? That’s the amazing thing - not that the feats themselve…

> this is not a linguist or a grade schooler, but a general purpose computer program This is sleight of hand. The training data almost certainly includes thousands of grade schoolers and linguists. Your argument seems to be that this is all generated ad nihilum from first principles (it's "just" a computer program bro), but that's not how it actually came to be. Rather, it was trained on literally terrabytes and terr…

But aren't most linguists trained similarly? Broad spectrum knowledge throughout their elementary education and specifics in higher education? Mostly regurgitating things they have either memorized or been shown repeatedly.

Re: I Taught ChatGPT to Invent a Language

#95

Post by ogogmad »

dvt wrote: post #85
dwohnitmok wrote: post #62

Earlier quoted context omitted.

I dunno. The people who I talk to who do seem to understand what's happening seem even more impressed than the average layperson (mainly that such a fundamentally simple technique is paying off with such crazy dividends).

> mainly that such a fundamentally simple technique is paying off with such crazy dividends I will concede that I am amazed by this. Even more generally, I'm even more amazed by Markov chains: an even simpler technique that can generate crazy dividends as well.

[edit: Was snark. Sorry. I also may have misunderstood the parent comment.]

Re: I Taught ChatGPT to Invent a Language

#97

Post by fnordpiglet »

PaulHoule wrote: post #80
fnordpiglet wrote: post #79

Earlier quoted context omitted.

I’ve been using it to teach me programming languages. If it plateaus here I’m delighted by its uniqueness and utility. You seem to think people are hinting this is general ai. I don’t think that’s what’s amazing people. I’ll bet you Siri and Alexa would be 100x more useful with a language model like this behind it. Part of their uselessness is the inability to generalize or reason, but relying instead in coded prompt…

"100x more useful" is pure hyperbole. 2x maybe, but 100x is a symptom of exactly what's wrong about people's thinking about ChatGPT. I would point to "There is no royal road to geometry" https://en.wikiquote.org/wiki/Euclid high-social status people have often been put off by quantitative reasoning because you spend years learning to do it, spend time gathering facts, thinking hard about things and often end up with…

Have you ever used Alexa? 2x more useful than alexa is as useful as my doorstop. It can’t understand most prompts, and defaults to reading a snippet from Wikipedia on some random subject then starts playing a pop song from Malaysia at volume 10 while ordering 60,000 incandescent black light bulbs with 2 hour shipping.

Re: I Taught ChatGPT to Invent a Language

#98

Post by whymauri »

fullstackchris wrote: post #53

I see posts like this and don't understand how they are made. I asked chatGPT this morning if it "remembered" something I asked in the previous prompt, and it said something to the tune of "I am simply a NLP machine learning model, I provide a decent output for good inputs, I don't have a sense of context, memory, or time." That's why I really don't understand these contextualized multi step conversations. Especially…

I asked it to simulate a Chess engine and it gave the usual "I am a blah blah blah, I cannot blah blah blah, because blah blah." I clicked "Try Again" and it worked.

Re: I Taught ChatGPT to Invent a Language

#99

Post by pmontra »

ogogmad wrote: post #60

Introducing: AI-Oriented Programming. Coming soon. If you allow a tool like ChatGPT access to APIs, I think it opens up a new programming paradigm: AI-Oriented Programming. The idea's that you talk to it, and tell it what to do in response to inputs. You then test out its ability to do those things, and correct its mistakes. There's no actual "code". In a way, everything that's hard about programming is because compu…

I think that the computers on the other side of the APIs won't understand ChatGPT's intentions too. Maybe ChatGPT would save us some time translating from our requirements to the right API calls. If it works like any of us, there will be meetings to explain requirements to ChatGPT, or a bunch of them, and more meetings to understand what they got wrong, why, how to workaround and how not to make it happen again.

Re: I Taught ChatGPT to Invent a Language

#100

Post by agravier »

seydor wrote: post #51

Earlier i was saying it isn't scary but maybe it is. It is impressive how ChatGPT keeps track of state and the flow of the conversation. How does it do it? Is it some kind of memory or some other model? ( I asked it, it keeps lying to me that it does not have memory)

Stateless. The whole conversation is replayed in the input.
Post reply on HN