Teaching ChatGPT to speak my son’s invented language
szopa.medium.com
Teaching ChatGPT to speak my son’s invented language
1–10 of 212 posts
Re: Teaching ChatGPT to speak my son’s invented language
#2Yeah I was think yesterday maybe we can start translating dolphin language.
Someone get on that
Re: Teaching ChatGPT to speak my son’s invented language
#3Super curious, would fine tuning with LoRa on a LLaMa/Alpaca model work better?
Re: Teaching ChatGPT to speak my son’s invented language
#4I would like to see this expanded, I think it's a bit unfair to assess its abilities with so few examples. My hypothesis is that a rosetta stone with a thousand examples with a vector database hooked up to it so you don't hit the 32k token context limit would lead to much better performance.
Re: Teaching ChatGPT to speak my son’s invented language
#5Yeah I was think yesterday maybe we can start translating dolphin language. Someone get on that
Someone is on that! https://time.com/6240144/aza-raskin-ai-animals-social-media/
Re: Teaching ChatGPT to speak my son’s invented language
#6Not trusting the models's self-assessment is the right call, considering that the actual score summed up to 7.5 compared to the self-reported 6.5 :)
Re: Teaching ChatGPT to speak my son’s invented language
#7Once again illustrating that the powerful thing about ChatGPT is that no matter what you do, it does its best to play along. Its eyes do not glaze over.
Re: Teaching ChatGPT to speak my son’s invented language
#8I would like to see this expanded, I think it's a bit unfair to assess its abilities with so few examples. My hypothesis is that a rosetta stone with a thousand examples with a vector database hooked up to it so you don't hit the 32k token context limit would lead to much better performance.
We'd love to see that too! However, I'm afraid that creating a substantial number of examples would transform this delightful family activity into something akin to punishment. Kłeti is quite the challenge for us Indo-Europeans, and it seems that even its creator isn't immune to the struggle.
Re: Teaching ChatGPT to speak my son’s invented language
#9In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before.
See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066
GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.
Re: Teaching ChatGPT to speak my son’s invented language
#10I would like to see this expanded, I think it's a bit unfair to assess its abilities with so few examples. My hypothesis is that a rosetta stone with a thousand examples with a vector database hooked up to it so you don't hit the 32k token context limit would lead to much better performance.
Both GPT-3.5 and GPT-4 versions of ChatGPT are limited to 4k tokens, even though GPT-4 is capable of 32k.
This leads me to believe that part of the reason for some of the mediocre results OP saw was because they hit the token limit and ChatGPT started "forgetting" earlier parts of the conversation.