301–307 of 307 posts
Earlier quoted context omitted.
I'm assuming this comment was written by ChatGPT, am I correct? It's got quite a predictable writing style, and also your comment doesn't add anything to the original, it's just reworded
yes :)
Earlier quoted context omitted.
It kind of feels that ChatGPT is "just" missing some kind of adversarial self clone that can talk back to itself and spot errors. Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model.
> Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model. By this do you mean that it correctly incorporated what you said and convincingly indicated that it understood the mistake? Because that's not the same thing as it having the truth latently encoded in the model—it just means that it knows how people respond when someone corrects them (which is usual…
"what's wrong with this code"
or "list 5 ways this can be improved,"
it does often recognize errors and give reasonable improvement suggestions.
Earlier quoted context omitted.
It kind of feels that ChatGPT is "just" missing some kind of adversarial self clone that can talk back to itself and spot errors. Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model.
> Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model. By this do you mean that it correctly incorporated what you said and convincingly indicated that it understood the mistake? Because that's not the same thing as it having the truth latently encoded in the model—it just means that it knows how people respond when someone corrects them (which is usual…
Earlier quoted context omitted.
Yes, exactly. There's also mathematically excellent reasons why that happened. Self-driving cars are an impossibly complex problem. Statistics are statistics. Predicting the minority class correctly 99% of the time isn't good enough for autonomous driving. A car has to break for little Suzie 100% of the time. However, generating 1,000 lines of code for a CRUD app? That's 99% bug free? That's a helluva lot better than…
The equivalent is closer to a CRUD app that serves 99% of requests correctly. Which is nowhere near good enough to use. But even if we do go with 99% bug free for the sake of argument, the usefulness depends on the type of bug. How harmful is it? How easy is it to detect? I had my wife (a physician) ask ChatGPT medical questions and it was almost always subtly but dangerously and confidently wrong. It looked fine to…
...knowing my first employer out of college, they'd be VERY happy with 99%.
"The end user will test the software" is very much the adage of the age. Cut priced, low quality.
Earlier quoted context omitted.
Ive subjected the thing to a professionally used IQ test and technically it scored very high on what I could test it at, however one way it lost points was claiming milk was transparent
What kind of tests did you feed it specifically? I’m curious to experiment with this too.