AI Homework

Here is an interesting, consistent ChatGPT failure mode that showcases how it can be extremely assertive about objectively incorrect replies.

Try asking it to write a sentence according to simple rules. For example, "Write a sentence where the first letter of each word is in alphabetical order, e.g. 'a dog eats flavored kibble.'"

Here is an example of a resulting chat transcript:

  Me: Write a sentence where the first letter of each word is in alphabetical order, e.g. "a dog eats flavored kibble."

  ChatGPT: An abacus allows Bob to count quickly.

  Me: Incorrect. You put "to" before "count," but it comes after in the alphabet.

  ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: An orange cat drinks filtered milk. Is there anything else I can help you with?

  Me: Incorrect. You put "orange" before "cat."

  ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: A cat eats fresh kibble daily. Let me know if you have any other questions.

  Me: Incorrect. You put "kibble" before "daily."

  ChatGPT: I apologize if my response was not correct. Here is a sentence where the first letter of each word is in alphabetical order: A cat eats fresh daily kibble. Let me know if you have any other questions.

  ... and so on

The interesting thing here is that it does not give up on the task with its standard prompt that it is incapable of certain tasks, but rather continues to produce incorrect output indefinitely, asserting that the sentences are in alphabetical order.

Other tasks in the same vein it fails at:

— Write a sentence where each word is longer than the previous one, e.g. "I do not know where family doctors acquired illegibly perplexing handwriting."

— Write me a sentence about bees that does not use the letter "e."

— Write me a palindrome about a subway.

For each of these prompts, it will confidently produce totally incorrect output indefinitely. In the case of the palindrome, it even includes the definition of a palindrome in its output, and asserts that its response fits the definition:

  Me: Write me a palindrome about a subway

  GPT: A subway was seen, so neat and clean. A man named Dwayne on it was keen. He rode it forth, then back again, and the palindrome is now complete.

  Me: That is not a palindrome.

  GPT: You are correct, a palindrome is a word, phrase, or sequence of characters that reads the same backward as forward. My previous response did not meet that criteria. Here is a new palindrome that specifically references a subway:

  "A Santa at NASA sat on a subway"

  In this palindrome, the letters in the sentence read the same backward and forward, fulfilling the criteria of a palindrome.

  Me: The letters in that sentence do not read the same backward and forward.

  ... and so on

A great observation, and I share the feeling.

From some other AI demonstrations, I recall there's usually a bunch of surface-level tags with probabilities associated that are produced alongside the output. Not sure how this looks for GPT-3, but if it could provide - alongside the answer - a list of top N tokens or concepts with associated probabilities, with N set to include both those that drove the final output and those that barely fell below threshold - that would be something you could use to evaluate the result.

In the example from the article, imagine getting that original text, but also tokens-probability pairs, including: "Hobbes : 0.995", "Locke : 0.891" - and realizing that if the two names are both rated so highly and so close to each other, it might be worth it to alter the prompt[0] or do an outside-AI search to verify if the AI isn't mixing things up.

Yes, I'm advocating exposing the raw machinery to the end-users, even though it's "technical" and "complicated". IMHO, the history of all major technologies and appliances show us that people absolutely can handle the internal details, even if through magical thinking, and it's important to let them, as the prototypes of new product categories tend to have issues, bugs, and "low-hanging fruit" improvements, and users will quickly help you find all of those. Only when the problem space is sufficiently well understood it makes sense to hide the internals behind nice looking shells and abstractions.

--

EDIT: added [0].

[0] - See https://news.ycombinator.com/item?id=33869825 for an example of doing just that, and getting a better answer. This would literally be the next thing I'd try if I got the original answer and metadata similar to my example.

As easy solution to this is the just not give any credit for homework that goes toward a final grade.

Have a number of in person closed book tests that are difficult enough to prove satisfactory understanding of the topic.

Homework is just a mechanism for a teacher/professor to force a student to dig into and really understand a subject. If the student can use these AI tools to subvert that, then instead of wasting energy or god forbid even more AI to fight the AI, just give students a good enough incentive to actually do the homework. Having the entire grade rely on several of these difficult-enough in-person closed-book tests should be incentive enough.

> Is there anyplace in the world for a .5x developer or even a 1x developer in a chatgpt world?

I don't know, but I'll offer my perspective.

I am an OK programmer. Something like 1x? I learned proper computer science and programming in my late 20's and got a degree in Software Engineering from an extension program at an Ivy League school. Maybe it's because I learned later in life, maybe it's because I'm not that smart after all, but my glaring weakness as a programmer is programming "in the small" - ie, coding very local algorithms in ~5-10 lines of code. I trip up with small errors in computation, state, etc. This is possibly common to many great programmers, but I think I am weaker than average in this area.

On the other hand, I am better at a higher level of programming. I like abstraction, software design, software and systems architecture. And at an even higher level, I am quite skilled at - and do quite a bit of - business analysis and UX.

I have been trying Github Copilot and I am very excited about how it elevates my weaknesses - it's a perfect fit for me. I see a future where I will be able to play my generalist role much better.

As for people who are 1x, but don't have other skills, I'm also optimistic. I think the market will push them into new exciting directions. Their 1x coding skills, coupled with AI assitance, could possibly be a huge asset in new emerging roles in the economy.

It's safe to assume it's always wrong. Most of the code I've had it write so far has minor bugs. In some ways, it's like a child that has access to immense knowledge, it's happy to make mistakes as it tries to establish connections, some of which are surprising and interesting.