Live data from Hacker News

I Taught ChatGPT to Invent a Language

maximumeffort.substack.com

101–110 of 250 posts

Re: I Taught ChatGPT to Invent a Language

#101

Ask #ChatGPT to list all ways to do X If it misses a way you know then ask if it knows about it If it confirms, ask it why it didn't include it In my case, it crashed (on two such attempts with different questions, happy to share) It seems to recall answers it has seen but not able to expand on those answers with knowledge it already has That's because it does not have knowledge. It has a bunch of correlations, encod…

What do you mean by "it crashed". No output?

Re: I Taught ChatGPT to Invent a Language

#102
post #89

Earlier quoted context omitted.

ChatGPT's superpower is hijacking the human desire for meaning into causing people to believe it is much better than it really is. https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach has dialogues between Achilles and the Tortoise where they work out the limitations of various fantastic A.I. systems based on mathematical logic. Roughly, neural networks don't repeal the results of Gödel, Turing and Tarski. The shor…

I had a weird effect with it: I played around with it for about an hour and was pretty impressed - I can't really wrap my head around how it can do some of the things it does. Then I started to see the patterns, and got repeatedly disappointed by having to adjust and refine my prompts. Then I got downright nonsense back from it and gave up. Either I made it to some kind of uncanny valley, and it would get better if I…

I feel like it’s state can be corrupted and it ends up in some weird rut. Often by telling it to try again I can unwedge it but sometimes I have to reset the entire state and start the convo over again. Each time is slightly different. These seem like bugs that optimization heuristics can overcome (like maybe replaying the same convo repeatedly and choosing the highest information next prompt to present? Who knows. It’s basically magic and I’m waving coconuts hoping the planes will land)

Re: I Taught ChatGPT to Invent a Language

#103
post #84

Earlier quoted context omitted.

ChatGPT's superpower is hijacking the human desire for meaning into causing people to believe it is much better than it really is. https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach has dialogues between Achilles and the Tortoise where they work out the limitations of various fantastic A.I. systems based on mathematical logic. Roughly, neural networks don't repeal the results of Gödel, Turing and Tarski. The shor…

That wasn't Hofstadter's point. Hofstadter didn't try to "disprove" the possibility of having AI -- he only set out to show that certain approaches couldn't work. GPT's approach isn't one of those he argued against. You seem to be referring instead to Penrose and Lucas, but their arguments have convinced virtually no one.

My point is that you can relax constraints such as Gödel's incompleteness theorems if you are going to give it credit when it gets wrong answers.

It might be able to write a TV Show like Sliders but it won't be able to do real physics, for instance.

Re: I Taught ChatGPT to Invent a Language

#104

I see posts like this and don't understand how they are made. I asked chatGPT this morning if it "remembered" something I asked in the previous prompt, and it said something to the tune of "I am simply a NLP machine learning model, I provide a decent output for good inputs, I don't have a sense of context, memory, or time." That's why I really don't understand these contextualized multi step conversations. Especially…

ChatGPT remembers the things it was trained on, and things that came up in the last 2048 tokens of dialog, which is about 600 words. (2048 is the GPT-3 size and I'm only guessing that ChatGPT uses the same size.)

Re: I Taught ChatGPT to Invent a Language

#105

Ask #ChatGPT to list all ways to do X If it misses a way you know then ask if it knows about it If it confirms, ask it why it didn't include it In my case, it crashed (on two such attempts with different questions, happy to share) It seems to recall answers it has seen but not able to expand on those answers with knowledge it already has That's because it does not have knowledge. It has a bunch of correlations, encod…

What do you mean by "it crashed". No output?

At first, there was no output. After maybe 3-4 minutes it displayed "An error occurred ... contact help etc" message. I waited about 30 seconds then tried again and it said that was too fast. So I waited a whole minute and tried again and it immediately spat out the "An error occurred" message. To me, the idea that it crashed stuck with me because of how it streams its output, which was empty for 3-4 minutes on the first try.

EDIT:

To clarify, based on the above sequence of events and timings, I figured the inference instance crashed and the API layer returned and error then cached it.

Re: I Taught ChatGPT to Invent a Language

#107
post #51

Earlier i was saying it isn't scary but maybe it is. It is impressive how ChatGPT keeps track of state and the flow of the conversation. How does it do it? Is it some kind of memory or some other model? ( I asked it, it keeps lying to me that it does not have memory)

Pretty sure it gets the entire conversation as input

Re: I Taught ChatGPT to Invent a Language

#108

As a committed conlanger (but not a good one: I didn't land the Dothraki contract), I read this article with some trepidation. Is my weird little hobby soon to fall victim to the oncoming AI tsunami? After the read, I find myself pleasantly surprised. My initial suspicion that the language would turn out to be Just Another Relex (make a new language by replacing English words with invented words, keeping the grammar…

No post body was provided.

Re: I Taught ChatGPT to Invent a Language

#109
post #89

Earlier quoted context omitted.

I had a weird effect with it: I played around with it for about an hour and was pretty impressed - I can't really wrap my head around how it can do some of the things it does. Then I started to see the patterns, and got repeatedly disappointed by having to adjust and refine my prompts. Then I got downright nonsense back from it and gave up. Either I made it to some kind of uncanny valley, and it would get better if I…

I feel like it’s state can be corrupted and it ends up in some weird rut. Often by telling it to try again I can unwedge it but sometimes I have to reset the entire state and start the convo over again. Each time is slightly different. These seem like bugs that optimization heuristics can overcome (like maybe replaying the same convo repeatedly and choosing the highest information next prompt to present? Who knows. I…

Pretty much! It probably sounds lame, but the main reason I enjoy programming is that I can, if I take enough time, reason about everything that's going on and use that knowledge to build, fix and improve systems people get some value out of. For me, the magical part is the possibility to understand the magic.

I find it even hard to reason about models I have full access to, downright impossible if it's some blackbox on someone else's severs.

Maybe I'm outdated, I suppose time will tell.

Re: I Taught ChatGPT to Invent a Language

#110
post #95
post #85

Earlier quoted context omitted.

> mainly that such a fundamentally simple technique is paying off with such crazy dividends I will concede that I am amazed by this. Even more generally, I'm even more amazed by Markov chains: an even simpler technique that can generate crazy dividends as well.

[edit: Was snark. Sorry. I also may have misunderstood the parent comment.]

What a weirdly snarky comment (it's like your third one in this thread, maybe you need to take a breather). I wasn't comparing them, though there's an obvious stochastic argument to be made; I was just commenting on the former's lack of complexity and still fascinatingly good results, I'm not sure what your point is.
Post reply on HN