What ChatGPT and AI-Based Program Generation Mean for Future of Software

Knowing what you want is both the fun and the hard part of most software development. That and understanding the environment you are operating on. Both of these things are not something I feel an AI has a strong advantage with given the fact that you need to provide this information.

Perhaps I can try to say, "Please decide what browser features I should use to maximize profit.", and it will actually do some proper research and contract out studies and all that... but I don't see that happening yet. Even still, the point of validation remains.

I can't help but draw a parallel with automated theorem provers. Sure they resolve "true", but then who validates the validators? It's a never ending cycle.

At the end of the day, you must learn to love and trust your tools and then stand by the joint creations.

Some years ago (perhaps 2015) I told a non-programmer that MOST programmers would be obsolete by the end of the century because of AI. Surprisingly he scoffed saying there is no way AI will be able to handle all those business edge cases. The stuff in the article is pretty basic and a long way from complex business logic, but we are on our way. I just hope business application developers like myself can survive until retirement, which for me is sometime around 2050. I am getting less optimistic.

[dead]

So... If it's so revolutionary, why can't I get it to solve level 1 advent of code problems?

Like here is what it generates for the 2016 day 1 problem:

def find_distance(instructions):

    x, y = 0, 0

    direction = 0 # 0: North, 1: East, 2: South, 3: West

    visited = set()

    visited.add((0,0))

    instructions = instructions.split(", ")

    for instruction in instructions:

        turn = instruction[0]

        distance = int(instruction[1:])

        if turn == "R":

            direction = (direction + 1) % 4

        else:

            direction = (direction - 1) % 4

        for _ in range(distance):

            if direction == 0:

                y += 1

            elif direction == 1:

                x += 1

            elif direction == 2:

                y -= 1

            else:

                x -= 1

            if (x, y) in visited:

                return abs(x) + abs(y)

            visited.add((x, y))

    return abs(x) + abs(y)

This function returns 113 from my input for that day, which is actually the answer for part 2... For part 1 it should be 234.

When I tried in Rust the solution didn't even compile, which is business as usual as far as my experience goes for trying to get ChatGPT to write anything practical (not a 'toy' example) in Rust.

I gave it another chance with day 2 in python and it failed at that as well. These are VERY simple tasks, CHILDREN can solve the initial couple days of advent of code.

In this article they give an example of a square root function. Maybe the authors could consider trying some more realistic tasks? So silly...