Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching ChatGPT to speak my son’s invented language

szopa.medium.com

Post reply on HN
Print view View original

71–80 of 212 posts

Re: Teaching ChatGPT to speak my son’s invented language

#71

Post by elhudy »

zenlikethat wrote: post #69
elhudy wrote: post #65

Earlier quoted context omitted.

In-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commercializing it in a B2B fashion. ...Or maybe that's coming? Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?

D2C makes a lot of sense imo because at this point it's most valuable to obtain a huge corpus of (reasonably well labeled) training data (with a side helping of being really great marketing). Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something…

That's a great point. As someone who isn't really a programmer I wouldn't have seen the value in that feedback until you pointed it out.

Re: Teaching ChatGPT to speak my son’s invented language

#72

Post by zenlikethat »

dgritsko wrote: post #46

The idea of asking it to produce an "ouroboros prompt" that can be fed back into itself summarizing everything already learned is very clever; definitely going to use that in future ChatGPT sessions of my own.

It's surprisingly good at compressing and decompressing even sophisticated information if you ask it to! Makes you realize how much of our words are pretty much just fancy padding.

Re: Teaching ChatGPT to speak my son’s invented language

#73

Post by crdrost »

Wow, they asked the model to self-evaluate and it just outright cheated:

    He has three cats.
    Proposed: h’io’ngkiltrikumrikumrikumri’nguuy
    Correct: h’io’ngkiltri’ngkumrikumri’nguuy
    Points: 1
    Hypothesis: N/A
(Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d'oh)

I have had similar problems with trying to get ChatGPT to do nontrivial things, "here are the rules for this game, do you understand this game, great, let's play it." And then it's like herding cats. "No that's wrong, the game pieces cannot leave the game board," "Oh my apologies you are entirely correct, here is the revised board (proceeds to dump the exact same state of the game board that I told it was wrong)." Eventually it will lie about its own capacities, "As an AI language model I am incapable of selecting a move to play next"... But you have done several already!!! This is literally the ONLY thing you have been doing right and now you refuse?

Some other prompts are more successful but it does seem to have a sing-song high school book review style that inclines it to be boring... Very uncanny valley.

Re: Teaching ChatGPT to speak my son’s invented language

#74

Post by [deleted] »

chmod775 wrote: post #22
bko wrote: post #18

Earlier quoted context omitted.

You said you don't have access but based on your tests... Were you testing ChatGPT I just tried and it was able to play tic tac toe, reverse a string (the string was "hello world.i am new to this so please forgive me if i can't reverse a sentence") Hangman sort of worked but it said every letter I picked was correct and appears to have constructed a word based on my guesses. Very strange behavior

Try making it reverse this: "Quickly, the kangaroo hopped away, escaping under the azure sky." I couldn't make it reverse that correctly even after prompting it five times to fix its mistakes. Most commonly it writes: ".yks eruz a eht rednu gnipacse ,yawa depoh ooragnak eht ,ylkciuQ" It also can't find the mistakes in there for the life of it.

[deleted]

Re: Teaching ChatGPT to speak my son’s invented language

#75

Post by sharkweek »

crdrost wrote: post #73

Wow, they asked the model to self-evaluate and it just outright cheated: He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h’io’ngkiltri’ngkumrikumri’nguuy Points: 1 Hypothesis: N/A (Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d'oh) I have had similar problems with trying to get ChatGPT to do nontrivial things, "here are the rules for this ga…

I was trying for 20 minutes to get it to spit out all 50 state capitals with the city names in alphabetical order and it kept doing two things:

1) It'd put the list in alphabetical order by state, but it'd include all the correct capitals

2) It'd list 49 of the 50 capitals, in alphabetical order this time, but duplicating Madison, WI.

I'd ask it to try and figure out what it did wrong in both cases, and it'd correctly identify the mistake, but then repeat it.

Not sure how I got there eventually, but on about the 7th or 8th attempt, it got it right.

Re: Teaching ChatGPT to speak my son’s invented language

#76

Post by adventured »

crdrost wrote: post #73

Wow, they asked the model to self-evaluate and it just outright cheated: He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h’io’ngkiltri’ngkumrikumri’nguuy Points: 1 Hypothesis: N/A (Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d'oh) I have had similar problems with trying to get ChatGPT to do nontrivial things, "here are the rules for this ga…

There is some amusement in that if we manage to build various AGI, these aspects will tend to get drastically worse rather than better. They'll be exceptional at fraud, cheating, lying, stealing, manipulation. Far beyond anything humans have ever managed.

People think fake news is a problem today, they haven't seen anything yet.

The Europeans will move on it first and attempt to hyper regulate it to stop it or slow it down. They'll fail.

Re: Teaching ChatGPT to speak my son’s invented language

#77

Post by z3c0 »

elhudy wrote: post #65
famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

In-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commercializing it in a B2B fashion. ...Or maybe that's coming? Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?

It's still not an extractive tool (it's generative), so it stating logical facts based on the data is not something one should bank on. It will likely never be capable of what you're suggesting, without pairing it with an actually-extractive model. The GPT-4 paper outright states that it should not be used for sensitive or high-stakes matters, which is what a lot of enterprises would consider their data pipelines.

Re: Teaching ChatGPT to speak my son’s invented language

#78

Post by joshribakoff »

Buttons840 wrote: post #58
famouswaffles wrote: post #9

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching…

Being able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update the weights in real time.

Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot

https://www.technologyreview.com/2020/10/16/1010566/ai-machi...

updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”

Re: Teaching ChatGPT to speak my son’s invented language

#79

Post by Name_Chawps »

knome wrote: post #24
afro88 wrote: post #10

Earlier quoted context omitted.

Both GPT-3.5 and GPT-4 versions of ChatGPT are limited to 4k tokens, even though GPT-4 is capable of 32k. This leads me to believe that part of the reason for some of the mediocre results OP saw was because they hit the token limit and ChatGPT started "forgetting" earlier parts of the conversation.

GPT-4 allows you to use 8k of context in their current beta, if you're using the chat api directly. It will be interesting ( and probably expensive, lol ) when they open it to a full 32k.

Though unfortunately it will cost like $20 per 32k completion...
Post reply on HN