Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching ChatGPT to speak my son’s invented language

szopa.medium.com

Post reply on HN
Print view View original

21–30 of 212 posts

Re: Teaching ChatGPT to speak my son’s invented language

#21

Post by ollien »

simonw wrote: post #19
ollien wrote: post #17

Earlier quoted context omitted.

Well, for hangman at least, if the human knows the secret, it should be possible for the LLM to handle that, no?

Oh right - yeah, that would work great. I'd be very surprised if ChatGPT couldn't do that.

I asked Bing AI

> I am thinking of a five letter word. You guess the word letter-by-letter (in the style of the game hangman) as many times as you like, but may only guess incorrectly six times. Please begin guessing

> Sure! Let’s start with the first letter of the word you’re thinking of. What is it?

lol

but it did get the picture after that. It does seem to be very fixated on guessing vowels though, after it has already exhausted all of them.

Re: Teaching ChatGPT to speak my son’s invented language

#22

Post by chmod775 »

bko wrote: post #18
marcodiego wrote: post #12

I don't have access to ChatGPT4, but in my tests I could observe that it can't do some very simple tasks: - It can't play tic-tac-toe, - It can't play hangman, - It insists that winning on stone-paper-scissor using the chat (playing before me) is a matter of probability. It was also demonstrated that it can't reverse strings. Actually a transformer doesn't accesses 'strings', all it processes are tokens which are the…

You said you don't have access but based on your tests... Were you testing ChatGPT I just tried and it was able to play tic tac toe, reverse a string (the string was "hello world.i am new to this so please forgive me if i can't reverse a sentence") Hangman sort of worked but it said every letter I picked was correct and appears to have constructed a word based on my guesses. Very strange behavior

Try making it reverse this: "Quickly, the kangaroo hopped away, escaping under the azure sky."

I couldn't make it reverse that correctly even after prompting it five times to fix its mistakes.

Most commonly it writes: ".yks eruz a eht rednu gnipacse ,yawa depoh ooragnak eht ,ylkciuQ"

It also can't find the mistakes in there for the life of it.

Re: Teaching ChatGPT to speak my son’s invented language

#23

Post by simonw »

ollien wrote: post #21
simonw wrote: post #19

Earlier quoted context omitted.

Oh right - yeah, that would work great. I'd be very surprised if ChatGPT couldn't do that.

I asked Bing AI > I am thinking of a five letter word. You guess the word letter-by-letter (in the style of the game hangman) as many times as you like, but may only guess incorrectly six times. Please begin guessing > Sure! Let’s start with the first letter of the word you’re thinking of. What is it? lol but it did get the picture after that. It does seem to be very fixated on guessing vowels though, after it has al…

It took a bit of prompt engineering but this worked for me in ChatGPT v4:

> Let's play hangman. I have thought of a 8 letter word, you have to guess it one guess at a time. You get to draw an ascii-art hangman too. Then YOU guess with a letter and I'll tell you if you were right or not. I won't give you a category clue. Start by drawing the hangman in a code block and then guessing a letter.

The same prompt against 3.5 didn't work - it didn't seem to be guessing likely next letters, and it couldn't keep track of how many body parts it should have drawn on the diagram.

Re: Teaching ChatGPT to speak my son’s invented language

#24

Post by knome »

afro88 wrote: post #10
JCharante wrote: post #4

I would like to see this expanded, I think it's a bit unfair to assess its abilities with so few examples. My hypothesis is that a rosetta stone with a thousand examples with a vector database hooked up to it so you don't hit the 32k token context limit would lead to much better performance.

Both GPT-3.5 and GPT-4 versions of ChatGPT are limited to 4k tokens, even though GPT-4 is capable of 32k. This leads me to believe that part of the reason for some of the mediocre results OP saw was because they hit the token limit and ChatGPT started "forgetting" earlier parts of the conversation.

GPT-4 allows you to use 8k of context in their current beta, if you're using the chat api directly. It will be interesting ( and probably expensive, lol ) when they open it to a full 32k.

Re: Teaching ChatGPT to speak my son’s invented language

#25

Post by chankstein38 »

vintermann wrote: post #7

Once again illustrating that the powerful thing about ChatGPT is that no matter what you do, it does its best to play along. Its eyes do not glaze over.

One of the things that always gives me a little hit of hype is when I tell it to do something ridiculous and it just dutifully starts spitting out the result without complaining or questioning lol

Re: Teaching ChatGPT to speak my son’s invented language

#27

Post by chankstein38 »

marcodiego wrote: post #12

I don't have access to ChatGPT4, but in my tests I could observe that it can't do some very simple tasks: - It can't play tic-tac-toe, - It can't play hangman, - It insists that winning on stone-paper-scissor using the chat (playing before me) is a matter of probability. It was also demonstrated that it can't reverse strings. Actually a transformer doesn't accesses 'strings', all it processes are tokens which are the…

You're trying the old primary school method of testing things based on the wrong criteria. Why does it need to play tic-tac-toe, hangman, or rock paper scissors? Why do you think a language model would be good at those things?

Similarly, why would you expect a thing trained on the context of text would be good at reversing strings? It's amazing it's as good as it is at these things because it doesn't really make sense that it could do these things unless they trained it on reversed strings to add diversity but it's hard to gain context from a string backwards.

ALSO: your transcript doxs you and is hard to tell where your messages end and GPT begins. Just a heads up in case you don't want your full name leaked to whoever reads this

Re: Teaching ChatGPT to speak my son’s invented language

#28

Post by fcatalan »

I've been trying a few things, some are very interesting.

For example it understands Europanto* perfectly, but when I asked it to produce some it was germanic-only Europanto: English, German, Danish, Swedish... I told it to use more romance words and he came up with pure French. After some more prodding he achieved a decent mix.

I also tried to get it to behave like an ersatz Duolingo for Basque and it sorta worked, but it would need some clever working on the prompts to really be usable.

(*) Europanto is a joke language that uses random European language vocabulary on top of a generally English grammar.

Re: Teaching ChatGPT to speak my son’s invented language

#29

Post by skybrian »

famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

I agree that it’s a neat demo, but it’s not all that useful in itself. You could also do this by writing a function in a programming language (if it weren’t built in), to arbitrary accuracy, and it doesn’t cost anything to run.

A more practical thing to do for algorithms is probably to use ChatGPT to help you write the function you need.

Re: Teaching ChatGPT to speak my son’s invented language

#30

Post by famouswaffles »

marcodiego wrote: post #12

I don't have access to ChatGPT4, but in my tests I could observe that it can't do some very simple tasks: - It can't play tic-tac-toe, - It can't play hangman, - It insists that winning on stone-paper-scissor using the chat (playing before me) is a matter of probability. It was also demonstrated that it can't reverse strings. Actually a transformer doesn't accesses 'strings', all it processes are tokens which are the…

Tic tac toe (on GPT-4) works with this

https://pastebin.com/cPwpZnZu

Post reply on HN