Dijkstra's Crisis: The End of Algol and Beginning of Software Engineering (2010) [pdf]

"Software people are not alone in facing complexity. Physics deals with terribly complex objects even at the "fundamental" particle level. The physicist labors on, however, in a firm faith that there are unifying principles to be found, whether in quarks or in unified field theories. Einstein repeatedly argued that there must be simplified explanations of nature, because God is not capricious or arbitrary.

No such faith comforts the software engineer. Much of the complexity he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God."

- Fred Brooks, No Silver Bullet

Yes, who would say structured programming is bad these days, though maybe it wasn’t a popular sentiment at the time. Though I find this thread funny, “Dijktra was elitist, oh no, he did the gatekeeping, what horror, let’s abandon structured programming and seeing programming as a discipline altogether”.

We should always try to look beyond Dijkstra's sometimes inflammatory words and into the spirit of what he is trying to convey; that is where the real value lies.

Excerpts:

What, for Dijkstra, was the crisis? I believe it was the separation of practice from theory in the development of commercial systems software. Recall the background shared by many of the most distinguishing contributors to the Nato conference, such as Dijkstra, Hoare, Perlis, Naur, and Wirth. They were exceptionally gifted men, many with strong scientific backgrounds, who in the early- and mid- 1960s had produced effective, elegant and reliable pieces of systems software with small teams and tiny budgets while working in research environments or for marginal computer firms. They were not by temperament successful corporate managers or project management specialists. So their instinct was to recoil as word spread of out-of-control industrial development efforts in which millions were being spend to produce unreliable, bloated, and incomplete operating systems and compilers ... Instead their faith was that with the application of suitably rigorous and mathematical methods a small team could produce high quality systems software without the need for massive teams of programmers and elaborate bureaucratic controls.

Near the end of his life, Dijkstra wrote that

"The ALGOL implementation was one of my proudest achievements. Its major significance was that it was done in 8 months at the investment of less than 3 man-years. The rumour was that IBM's FORTRAN implementation had taken 300 man-years and thus had been a project in size way beyond what a university could ever hope to undertake. By requiring only 1 % of that labour, our compiler returned language implementation to the academic world, and that was its political significance".

Yet his single biggest complaint, and one underlying many of the others, was against the idea that computer scientists must adjust their expectations to a world in which most programmers are intellectually mediocre and theory-averse. This is why he called himself a “humble programmer” in his Turing award speech, rather than, as his colleagues might, an arrogant (albeit insecure) academic computer scientist. Trained as a physicist he believed that the proper model for programming came from “mathematical engineering,” a concept he claimed was well understood in the Netherlands but which thoroughly confused his American colleagues. By seizing the mantle of programmer, despite his increasingly deep immersion in the world of mathematical theory and formal methods, Dijkstra could insist that he had not left programming behind. If actual programmers failed to grasp the importance of his new work then it was they who were betraying the true essence of programming.

Dijkstra gave a particularly clear statement of this in 1977, in an open letter circulated to fellow members of Working Group 2.3. According to Dijkstra, in the late 1960s it “became abundantly clear that we did not know how to program well enough.” As a result “people concerned with Programming Methodology,” by which he presumably means him and his fellow working group members, discovered that programming was “a tough engineering discipline with a strong mathematical flavor.” However this inarguable conclusion was sometimes ignored, “because of the unattractiveness of its implications, such as,”

(1) good programming is probably beyond the intellectual abilities of today's 'average programmer’

(2) to do, hic et nunc, the job well enough with today's army of practitioners, many of whom have been lured into a profession well beyond their intellectual abilities, is an insoluble problem

(3) our only hope is that, by revealing the intellectual contents of programming, we will make the subject attractive to the type of students it deserves, so that a next generation of better qualified programmers may gradually replace the current one.

According to Dijkstra, only “political or emotional reasons” could explain disagreement with these “technical” observations. He conceded that they were “unattractive” and had “severe” social implications and no quick solutions. Clearly it would not be easy to replace almost the entire programming workforce. But he dreamed of a scientific revolution, in which programming would win recognition as a mathematical discipline and the incompetent hordes currently messing things up would be swept away and replaced by an elite corps of scientists versed in new and more formal methods of software development.

PS: With AI tools becoming the norm in programming today, things are going to become exponentially worse if "the average programmers" do not study and use formal mathematical methods for code generation and verification.

I was present at the IFIP 1978 conference in Toronto where Dijkstra announced that personal computing was a dead end because most people don't know how to program.

There's an irony about the design of ALGOL 68. It was denounced at the time as being too big a language, with too many experimental features. In fact, the language in the Report is unimplementable, primarily because the lexical syntax is fanciful (there's an actual difference between the roman period and the italic period) and also abstract (so no two implementations actually have the same lexical syntax). I was on the periphery of an ALGOL 68 implementation project that foundered because the goal was to implement every conceivable edge case, include those that could never occur in practice, e.g., x[i] := x[...50 pages of code...] Yet very reasonable subsets (including ALGOL-68R and FLACC) were built. As for the language breadth, arguably C# is much broader, and I haven't heard anyone claim that it is a failure.

As for the Report...there was a time when I understood it cover to cover. But Lindsey and van der Meulen's Informal Introduction presented the whole language at a level understandable to anyone who understood ALGOL 60. van Wijngaarden did the language a terrible disservice when he forced (that is the correct word, I believe) the use of two-level grammars in order to ensure type correctness. Had the Report never existed, the language would have had much more traction.

The irony? Apart from the grossly-overengineered “transput” library—much better adapted to a unit-record and line printer world than our modern world—the one actual experimental feature in the language was the par clause, which provided parallel execution, and relied on semaphores for synchronization. Dijkstra's semaphores.

No, I feel like software developers are unable to predict the future. Mathematics and logic aren't much help with that and physics barely enters into it.