Live data from Hacker News

Does the software industry learn?

tomrenner.com

51–60 of 102 posts

Re: Does the software industry learn?

#51

I think there's a myth that rewriting software is bad, or at least a symptom of badness, and "reusing" software is ideal. But writing software is really like creating plans: plans for what the machine should do. Of course there is much room for reusing old plans as components of your new plan. But still every plan must be about what is needed at the moment, not about reusing existing plan-components. In human communi…

From Alan Perlis' Epigrams[0]

> 14. In the long run every program becomes rococo - then rubble.

And;

> 15. Everything should be built top-down, except the first time.

I think these are helping to understand the lifecycle of a project.

A first top down design should be understood as a draft, a direction. With experience these drafts get better, but the act of bottom up programming is necessary to discover details, problems and requirements that inform a more robust and flexible design.

A rewrite can be partial or total. Maybe it is enough to refactor some components, maybe a new design was discovered during the initial iterations, which yields a system that composes and evolves more robustly and works better as a whole.

As a web developer I program user facing applications, so there is always a kind of pressure to have highly domain and feature specific interfaces. Just get it done, move to the next.

But by learning from the Lisp world [1], Out of the Tar Pit [2], software design books [3][4] and the pain of experience, I've come to the conclusion that there is a better way. There is often a general layer that wants to be discovered that we can use to build the needed features. A workflow, framework, DSL or maybe just a bunch of functions or a data model.

This layer is often very hard to get right the first time around, especially if we don't have strong analogies already. It is often more natural to discover it by writing or trying to write the specific code first. Premature abstraction is the root of all evil.

But also [0]:

> 7. It is easier to write an incorrect program than understand a correct one.

We should be brave enough to move forward, while assuming that we're probably wrong. Scientists often practice this principle and try to figure out in what ways they _are_ wrong. I think this is a good way to approach programming too.

[0] http://www.cs.yale.edu/homes/perlis-alan/quotes.html

[1] https://dspace.mit.edu/handle/1721.1/6064

[2] http://curtclifton.net/papers/MoseleyMarks06a.pdf

[3] https://web.stanford.edu/~ouster/cgi-bin/book.php

[4] https://mitpress.mit.edu/books/software-design-flexibility

Re: Does the software industry learn?

#52

Given the number of times we have failed to learn the lesson "downloading code from untrusted sources and running it is a bad idea" -- the log4j and NPM colors fiascos spring to mind -- I think it's fair to conclude that this industry is completely incapable of learning anything, ever.

Not only that, but we're now creating devices that depend on remote servers. It's completely obvious what the downsides to this are, yet we embrace it without question. And why are moving to apps that only work online? Networks are slow compared to desktops. Programming seems to be the art of doing the worst thing possible. Our computers are getting faster and faster, but we're relying on communications that are vast…

I think it's preposterous to expect that software that has millions of line of code (like Windows or Firefox) is perfect. Software Engineers are humans as well and we do make mistakes.

You could argue that we should adopt the same testing strategy of mission critical pieces of logics where 1 LOC has 1+ LOCs of test code, aim for 100% coverage, ... . But then your windows license would cost 1000s of $.

And about your final remark. What you call real engineering has to cope with well defined requirements and the context is well known. You know beforehand the maximum weight a bridge should be able to support. You know the configuration and properties of the soil where the bridge will be built, and you have good estimates of 1 in 100 year extreme events magnitude.

Software doesn't have that luxury. If we build a web server to handle 1k qps it still is somewhat likely that said server might face spikes of 10k qps. Try to do that with the bridge mentioned above. Forget failing gracefully. Additionally users have few ways to use the bridge. You either cross it by car / bus / on foot. The same web server might face someone trying to send a payload of GBs where you would expect few KBs. That's at least partly why code is messy. The space of possibilities is much greater and we somehow need to write software that still works.

Re: Does the software industry learn?

#53

Earlier quoted context omitted.

You see, this sounds really cool. So what kinds of problems fit COBOL? And why did "we" conclude from this that general purpose languages were the solution, rather than building many different languages for specific problems? Did "we" even conclude that, or is that just my impression, 50 years later?

From what I remember of the times, it was the PC that killed COBOL. COBOL was considered a "serious" language, for "serious" business problems running on "serious" hardware that lived in a server room. PC's were not "serious", and lived on people's desks, running Lotus123 and other such trivial tools. Then VB/Delphi/etc came along, and a generation of developers writing "little" applications directly for the desktop…

> Writing an application in VB was several orders of magnitude cheaper

Order of magnitude is big deal, can you be more specific, did you mean 2 or 3 orders of magnitude? Or what exactly did you mean.

By this logic if writing an app in VB costs 20$ then COBOL equivalent cost 2000$ or 20 000$ ?

Ok, 20$ seems a bit too cheap, let's see, if developing VB app cost 400$. So COBOL equivalent would have cost either 40 000$ or 400 000$ ? Is that what you mean?

Re: Does the software industry learn?

#54
post #6

Earlier quoted context omitted.

Lookup YouTube on how many people built a chair, desk, or any other basic piece of furniture. In a craft you have to do the basics yourself sometimes.

And yet, those people do not get paid hundreds of thousands of dollars a year, believe their work to be at the height of economic efficiency, or think themselves generally more intelligent than the average person. Software engineering is not a craft. It is a discipline. If society is paying people these exorbitant sums to be the equivalent of digital Etsy moms, we're getting a raw deal.

Every discipline needs to evolve and Crafting is how you evolve it. The fact that the software industry evolves too fast is because the domain it's applied to is very large and thus we are constantly finding out problems that need crafting.

> And yet, those people do not get paid hundreds of thousands of dollars a year, believe their work to be at the height of economic efficiency, or think themselves generally more intelligent than the average person.

Designers can be payed more than a software engineer. And we don't know what they really think but there are plenty who think that they are better than anyone and are doing the society a big favor but people simply don't understand. (You don't have to go to far away from some software industry to find the most egregious examples of those :D)

Re: Does the software industry learn?

#55
post #21

> As a result of how young the profession is, there are few universally accepted practices and standards. I wish the meme of "software engineering is young" would die. This profession is not younger than any other technical profession, in the modern sense. It was born out of prior practice and is still taught using the same techniques and philosophy. We are not special. Furthermore, modern engineering is a mid to lat…

The software industry is young, if not in a literal sense then certainly in a figurative sense.

Other crafts define their professionalism according to a standard baseline defined by governance and credentials. Examples of credentials are licenses and certifications. Examples of other crafts that require credentials: medicine, law, engineering, flying, driving, real estate, information security, networking, food preparation, education, public securities, and many more. When this is brought up in software the result is excuses and whining, hardly an indication of maturity.

The problems with the industries exaggerated immaturity result in profuse subjectivity. There is always a perception, at hiring, that software developers are rare. Perhaps poorly trained or developed less than desired expectations but software developers are not rare. Nonetheless the problem is exaggerated in a cycle of attempts to increase availability among an undesirable candidate pool by abstracting away many challenges inherent to the practice. This is cyclical because instead of bridging a training or development gap it instead breeds entitlement and insecurity that is paid forward. Since there are no industry standards of acceptable practice this cycle becomes a race to the bottom.

This is the very essence of young.

Re: Does the software industry learn?

#56
post #34
post #31

Earlier quoted context omitted.

Why do most users need to contribute? The value is the ecosystem. The point is we don’t have to contribute to everything we use. We can build on the work of others and they can build on ours. Even developers of common libraries are relying on an amount of open code so immense they couldn’t possibly make contributions to all of it. This is the beauty of free software.

> We can build on the work of others and they can build on ours. This only works if both parties publish. Otherwise it's "we can build on the work of others"

Many companies seem to be able to benefit in various ways from contributing to open source / free software.

Examples: - Chromium and Android obviously benefits Google as it makes it easier to ensure adds get through - Also, they limit the ability of Apple/Microsoft to control those revenue streams in their walled gardens - Hardware and software vendors benefit from making sure Linux works well with their products - Making TensorFlow free helps build a community that in turn makes hiring easier. - Contributing to Torch may protect against a monopoly - Contributing to other R or Python machine learning tools may help limit the power of companies like SAS or IBM/SPSS - Similarly, contributions to Postgres/Mongo etc wrestles power away from Oracle, MS (MSSQL) and IBM (DB2). - More of the same: Proton vs DirectX, OpenCL vs Cuda, FidelityFX vs DLSS. When a competitor tries to establish a standard that is either paid for or limited or proprietary in some other way, providing or contributing to open alternatives may be easier to do than to provide a direct proprietary competitor. - DataBricks founders benefit from being part of the creation of Spark, and can get paid for adding further value.

Many of the above are cases where large to huge corporations use their power to disrupt competitors by providing free alternatives in areas where the has some market dominance. Other contributions assist in delivering a basic product for free while getting paid for products that add value on top.

Re: Does the software industry learn?

#57
post #6

Earlier quoted context omitted.

Lookup YouTube on how many people built a chair, desk, or any other basic piece of furniture. In a craft you have to do the basics yourself sometimes.

And yet, those people do not get paid hundreds of thousands of dollars a year, believe their work to be at the height of economic efficiency, or think themselves generally more intelligent than the average person. Software engineering is not a craft. It is a discipline. If society is paying people these exorbitant sums to be the equivalent of digital Etsy moms, we're getting a raw deal.

I don't know what software engineering is ans I strongly suspect you don't, either. I fact, I think that software engineering is just a term invented to give non-programmers (professors, managers, consultants) a somewhat technical career path. There is certainly no engineering happening when someone creates a new web service or a new device driver. I don't think there's engineering involved in Amazon's latest cloud service or googles latest feature.

Regardless, programming is a craft. And programmers are essentially crafts(wo)man. Everything's handmade, there's only unicates, there's no formal education besides some guild rules (aka coding interviews) and the science of computing which tells you how and why a program works or doesn't but not how you create it.

There has been some research into actual engineering practices for software. But the underlying science often gets ridiculed, not applied, by practical programmers.

Re: Does the software industry learn?

#58

I think learning is subjective. The NodeJS world is learning a lot about backend development I guess. So, disclaimer, I am new to Typescript-NodeJS and the npm world but have a significant background in the JVM / Spring backend world with Kotlin and Java. I am also too young to be this cynic but I am honestly just baffled. To me the entire NodeJS backend movement seems like they are "Java sucks!" hipsters jumping int…

As someone with a significant background in Node.js and a bit of experience in Java I think Node.js is much better in that respect.

Yes, we have things like NestJS, which I refuse to use precisely for the reasons that you point out: it looks like Java Spring and adds a lot of complexity where it is not needed.

I am also painfully aware that TS can be abused to write Java-like code but that's on the developer. However, the abuse notwithstanding, the fact that JS is a simpler language protects it from Java's key problem:

"Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle."

My (limited) exposure to Java was quite painful because I had to deal with layers and layer of inheritance and abstractions that at some point made it very difficult to do what I needed to do. The language itself was fine except that it easily allowed for this sort of architecture.

Also, the advantage of writing the frontend and backend in the same language should not be understated.

Re: Does the software industry learn?

#59

Earlier quoted context omitted.

My feeling is that rewriting is used as blunt weapon when the developer lacks the inclination or skill to learn what was written before. I’m been guilty of this.

But even if you know what is written, good luck making that PHP 3 website keep up with times. Or that old C library written for old versions of OS/other libs.

Agree, but you are giving a more extreme example.

Most rewrites I saw was "wow, this Java code from the old team is a mess, let's rewrite it and, in a couple years, the new devs will think the same about our rewrite."

Re: Does the software industry learn?

#60
post #15

Earlier quoted context omitted.

Unfortunately the browser implementations of the HTML5 datepicker only make sense for an audience that understands the US date notation. They can't be configured, as far as I know.

What? The html date picker just uses the system date picker. And the value is formatted as yyyy-mm-dd when you submit it, that's not the US date format.

After reading your reply I checked again and it does indeed work as expected on Windows at least, so that is great news to me.
Post reply on HN