Testing GPT 4's code-writing capabilities with some real world problems

Try again with chain-of-thought prompting?

Yes. I only need it to write 50-100 lines at a time to be incredibly effective.

My productivity this month has been insane. I'm still writing most of the code the old fashioned way, but the confidence of having this kind of tool makes it a lot easier to push through boring/tricky items.

I'm not sure that the commenter was negging here but instead showing that the language used in the prompt was imprecise (in a similar way that we might get system requirements in reality). The takeaway I got was more that the commenter was showing surprise at how well that ambiguity was resolved regardless. A more interesting experiment to run might be to intentionally reduce the precision of the task e.g. something

    In a 10x10 arena, work out if there is a path shorter than 5 steps from one start to finish with water tiles taking two steps and avoiding as many fire tiles as possible.

But you can go even better than that really. The author jumps down to a more code oriented description too soon (10x10, etc.) but ChatGPT (3.5) does ok with just the original problem (I removed the text about A* and Djikstra's from the english version of the problem statement:

PROMPT:

In Mewgenics, movement abilities use pathfinding to get the cat from his origin to his destination.

Cats have a maximum movement range stat and tiles have a cost (in this case its 1 for basic tiles and 9999 for blocking obstacles). We also have water tiles that cost 2 to path through.

The complication comes in when we add Fire (and other hazard type tiles) into the mix. Fire tiles don’t cost extra to pathfind through like water tiles do, however a cat really wants to avoid pathing through the fire tile if it can. If there’s a lot of fire tiles it should go through the least number of them it can.

ANSWER:

To solve this problem, you can modify the pathfinding algorithm to take into account the hazard tiles, such as fire. One way to achieve this is by using a modified A* algorithm.

First, assign a high cost to hazard tiles such as fire. This cost should be higher than the cost of regular tiles, but lower than the cost of blocking obstacles. For example, you can assign a cost of 5 to fire tiles.

Next, modify the heuristic function used in the A* algorithm to take into account the distance from the hazard tiles. For example, you can modify the Euclidean distance heuristic to add an extra term that takes into account the distance from the closest hazard tile. The closer the cat is to a hazard tile, the higher the heuristic value should be.

Finally, when selecting the next tile to visit, prioritize tiles with the lowest total cost (including both the movement cost and the heuristic cost), but also consider the number of hazard tiles that the cat would need to pass through to reach that tile. You can add an extra penalty term to the total cost that increases based on the number of hazard tiles the cat would need to pass through to reach the tile. This penalty term should be high enough to discourage the cat from pathing through too many hazard tiles.

With these modifications, the cat should be able to find the shortest path to its destination while avoiding hazard tiles as much as possible.

When I say singularity, I mean “changes the nature of human life irreversibly and to a greater degree than any prior technology through a rapid progression of self-reinforcing growth.”

That rapid progression can be modeled with an exponential growth curve, like Moore’s law, even though nothing material can have a sustained exponential growth. Regardless of how the curve is modeled, the steepness is such that it serves as a step function in evolution.

With that clarified, there is first the question of “is the technological progression going to level off soon?” And I think the answer is no. The second question is, “how are we going to deal?” And the answer to that question can only be addressed once we answer the question: “what kind of world do we want to live in?”

> I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile [...]

Well it is revolutionary. And it isn't just where it is today, but how fast these models are improving - with no saturation in ability evident at this time.

On the other hand, I am not sure anyone is saying programmers are infantile - although poorly written software is as at least as prevalent as poorly compensated or managed software development positions.