Thoughts on the Future of Software Development

Your coworkers likely aren't doing a very good job at reviewing, but also I don't blame them. The only way to be sure code works is to use it for its intended task. Brains are bad interpreters, and LLMs are extremely good bullshit generators. If the code makes it to prod and works, good. But honestly, if you aren't just pushing DB records around or slinging HTML, I doubt it'll be good enough to get you very far without taking down prod.

from min 1:29:55 — Sam Altman & Lex Fridman’s new interview, incl questions about AGI

https://youtu.be/jvqFAi7vkBc?si=tZXNdVnOSk1iWX34

I agree but there will likely be demand for experts to help direct AI to create a better solution than otherwise and even write pseudocode and math formulas sometimes. The key is for the expert to understand the user’s needs better than AI by itself and ideally better than the user themselves.

Many/most software engineers could act more like art directors rather than artists, or conductors instead of musicians.

But that's the thing, that it seems that everyone here on HN (and elsewhere) finds it easy to judge the flaws of AI-generated code, and they seem relatively consistent. So if we start offering these critiques as RLHF at scale, we should be able to bring the LLM output to the level where further feedback is hard (or at least inconsistent), right?

Speaking from an ethics point of view: at what point do we say that AGI has crossed a line and deserves self autonomy? And how would we ever know when the line is crossed?

"Automatic programming always has been a euphemism for programming with a higher level language than was then available to the programmer. Research in automatic programming is simply research in the implementation of higher-level languages.

Of course automatic programming is feasible. We have known for years that we can implement higher-level programming languages. The only real question was the efficiency of the resulting programs. Usually, if the input 'specification' is not a description of an algorithm, the resulting program is woefully inefficient. I do not believe that the use of nonalgorithmic specifications as a programming language will prove practical for systems with limited computer capacity and hard real-time deadlines. When the input specification is a description of an algorithm, writing the specification is really writing a program. There will be no substantial change from out present capacity.

The use of improved languages has led to a reduction in the amount of detail that a programmer must handle and hence to an improvement in reliability. However, extant programming languages, while far from perfect, are not that bad. Unless we move to nonalgorithmic specifications as an input to those systems, I do not expect a drastic improvement to result from this research.

On the other hand, our experience in writing nonalgorithmic specifications has shown that people make mistakes in writing them just as they do in writing algorithms."

Programming with AI, so far, tries to specify something precise, algorithms, in a less precise language than what we have.

If AI programming can find a better way to express the problems we're trying to solve, then yes, it could work. It would become a matter of "how well the compiler works". The current proposals, with AI and prompting, is to use natural language as the notation. That's not better than what we have.

It's the difference between Euclid and modern notation, with AI programming being like Euclidean notation and current programming languages being the modern notation:

"if a first magnitude and a third are equal multiples of a second and a fourth, and a fifth and a sixth are equal multiples of the second and fourth, then the first magnitude and fifth, being added together, and the third and sixth, being added together, will also be equal multiples of the second and the fourth, respectively."

a(x + y) = ax + by

You can't make something simpler by making it more complex.

https://web.stanford.edu/class/cs99r/readings/parnas1.pdf