Thoughts on the Future of Software Development

Humans also need human data. You might be better than I, but at least for myself, I know that I am just a weighted pattern matcher with a some stochasticity mixed in.

I don't think the idea of painstakingly writing out a book, and then having a printing press propagate your book so that all can easily reproduce the idea in their own mind, is so very different.

I think this is why the real conversation here is about the lossiness of the data, where the "data" is conveying a fundamental idea. Put another way, human creativity is iterative, and the reason we accept "innovative" ideas is that we have a shared understanding of a body of work, a canon, and the real innovation is taking the canon and mixing it up with one new innovation.

I'm not even arguing that AI is net good or bad for humanity. Just that it really isn't so different than the printing press. And like the Bible was to the printing press, I think the dominant AI model will greatly shape human output for a very long time, as the new "canon" in an otherwise splintered society, for good and for bad.

Proprietary models, with funding and existing reach (like the Catholic Church when the Gutenberg press came along), will dominate the mental space. We already have Martin Luther's nailing creeds to the door of that church, though.

Still, writing by hand does still have special meaning, encoding additional information that is not conveyed by printing press. But then as now, that additional meaning is mostly only accessible to those closest to you, that have more shared experiences with you.

I'll accept that there's an additional distinction, though, since layers of communication will be imported and applied without understanding of their context; ideas replaced, filled in, rather than stripped. But let's be honest: every interpretation of a text was already distinct and uniquely an individual's own, albeit likely similar to those that shared an in-group.

AI upsets the balance between producers and consumers, but not in the way that it's easier for more people to be producers, but in this day in age, that there is so little time left to be a consumer when everyone you know can be such a prolific producer.

Edit: typewriters and printing presses also need human data

Software Developers (humans) are the horses, the models are the mechanical cars, and the analysts and business experts are the drivers. Software's only job is to empower the end user, and that's exactly what these models are doing. We're going to see a massive paradigm shift in how software technology is built in the next decade.

My problem isn't the graph or even its particular failings (vanishing gradients in science being one of many), it's the ideological opposition to regularization.

> 1) Those tasks that it is still currently only possible for a human to do. 2) Those tasks which are easier and cheaper for a human to do.

I agree, but "1" must include all tasks where a mistake could lead to liabilities for the company, which is probably most tasks. LLMs can't be held responsible for their fuckups, they can't be punished, they have no body. It's like the genie from the bottle, it will grant your three wishes, but they might turn out in a surprising way and it can't be held accountable.

The same will apply for example for using LLMs in medicine. We can't afford to risk it on AI, a human must certify the diagnosis and treatment.

In conclusion we can say LLMs can't handle accountability, not even in principle. That's a big issue in many jobs. The OP mentioned this as well:

> even when AI coders can be rented out like EC2 instances, it will be beneficial to have an inhouse team of Software Developers to oversee their work

Oversight is basically manual-mode AI alignment. We won't automate that, the more advanced an AI, the more effort we need to put in overseeing its work.

I have seen programmers express that they dislike writing code before and I wonder what the ratio of people who dislike it and people who like it, is. For me, writing code is one of the most enjoyable aspects of programming.