Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do web sites and software take so long to build? And why is it so hard?

scottporad.com

Post reply on HN
Print view View original

31–40 of 159 posts

Re: Why do web sites and software take so long to build? And why is it so hard?

#31

Post by bigiain »

My opinion?

Because as much as the software industry loves to adopt the work "engineer", almost none of the sort of websites and software discussed here on HN ever gets anything much like "engineering" in the sense of, say, "civil engineering" done.

I think in my ~20year career, I've had only 3 projects that were specified well enough up front that we just "built it according to the plans" and had a satisfied customer at the end. Overwhelmingy, some (or most) of the design gets "made up as we go along". Even on projects with several small forests worth of up-front documentation, there's almost always large areas of vagueness or outright contradictory requirements, which need decision making on the developers part halfway into the job.

There are _very_ few websites that couldn't be coded from the ground up in a few weeks with a few good programmers - _IF_ you had already thought through all of the things the site needs and all of the consequences and the contradictions in those consequences. Even the big ugly complex projects, like Google's search or Facebook or Wikipedia or Twitter - if you had a spec that answered all the details about how you wanted them, a small team of experienced web guys could get it up and running in a month or so, and let you now how much cloud/hardware/sysadmin/support you'd need to budget for as your customer acquisition kicked in.

It doesn't take very long to "build" the software or the website (obvious "large scale" projects like OSes excepted). What takes most of the time is identifying and solving the problem. (And, I suspect a lot of the reason software/websites have a reputation for taking "so long" is that they're _way_ too often rushed into the "building" phase way before all the things that need designing are even identified, then all the new solutions take longer 'cause we try to fit them around all the work that's already been done.)

Re: Why do web sites and software take so long to build? And why is it so hard?

#32

Post by fierarul »

Nothing is made entirely by machines. At some point it's made by hand. The thing is software is a bit like doing clay pottery when it's not properly done. Once you add a bit of a systematic approach to it and you test is, review it, it becomes a bit closer to an industrial product.

The thing is the rest of the world is also in a state of flux. Plumbers changed materials quite a few times in the past decade which changed the fixtures a bit. So the concept is the same (ie. design patterns) but the actual material and its characteristics changed (ie. framework, programming languages) and there are different ways to put them together: welding, glue, etc (ie. APIs).

But changing some material, fixtures and binding material doesn't even compare to the kind of flexibility we have with software!

The solution for plumbers is that they always use what's current. Which makes their field as much of a fashion-driven field as ours (think about it).

But give them an existing house with multi-layer pipes and tell them to fix it and they will also comment that it's old tech and you should switch to copper or whatever.

In conclusion, I don't see how software is any different than other fields except that we are much more flexible and go with a much greater speed.

Software is at its infancy because science in general is at its infancy.

Re: Why do web sites and software take so long to build? And why is it so hard?

#33

Post by arethuza »

michaelneale wrote: post #23
maaku wrote: post #19

Except that software is written by machines these days. I stopped writing in assembly language after it stopped being cool in the 90's. Even still, you'd have to be hand-crafting machine code in a hex editor to really get all the machines out of the loop. These days I write in an interpreted language using complex libraries that handle a multitude of protocol choices for me. Even if you argue that each of those tools…

Exactly - this article makes the very common (still) (1) mistake of thinking construction of software == writing code. Construction of software is compiler/interpreter/all the machinery that kicks in to take some text or instructions and makes it do something. To correct the analogy with building physical things - you could compare the architect designing a bridge (and arguing with engineers about materials and budge…

The idea that code is the design and it's the compiler doing the construction has been around for quite a long time:

http://www.developerdotstar.com/mag/articles/reeves_design_m...

Re: Why do web sites and software take so long to build? And why is it so hard?

#34

Post by swecker »

I think of software development more as an art than a science. If those questions were asked of a professional painter I think we would hear similar responses. It's all by hand, colors shapes and textures can be amazingly complex in their combinations, standards would only inhibit a true artist, and a painter usually has no idea at all when a masterpiece might come forth. So my answer to "Why is is so hard?" would be, "because it's art." And "why do we keep doing it?" "Cause we're artists, and people enjoy art."

Re: Why do web sites and software take so long to build? And why is it so hard?

#35

Post by redwood »

tablatom wrote: post #24

To date, software engineering has failed to deliver on one critical goal, which is why we are in this mess. The world of software development needs to be divided in two -- the component creators, and the component assemblers. To stick with the given analogy, component creators are the people inventing new kinds of plumbing: easier ways of connecting pipes, taps that don't ever drip. Component assemblers are the peopl…

I think you're spot on. Just as the personal computer and GUI brought "computing" to the masses for their use and massive scale return on investment, we need to see the equivalent innovation in programming components. In other words just as it's now easier than ever to edit movies, write a paper, polish a photo: we need to see this ease of interface brought to programming.

I know hard core CS types will say that making programming easy for the masses will mean it won't be fast, it won't allow for the optimal algorithms. However computing speed and bandwidth is rising exponentially and countless applications for software do not need to be as optimal as possible.

In the future people who are not programmers but who are one step removed (e.g. okay with spreadsheets, SQL, some basic scripting when needed, visual programming like Labview, and who understand software architecture) will need to be able to do more complex things with computers, things that today only programmers can do.

We need to see the separation between programmer and the technical creative masses disappear a bit. Or at least as tablatom says, there has to be room for two types: those who build the tools for easy programming and those who do easy programming.

Re: Why do web sites and software take so long to build? And why is it so hard?

#36

Post by ExpiredLink »

Is it?

> meta name="generator" content="WordPress 3.0.1"

Was it hard to create this blog entry? Obviously he has a point but, in general, the abundance of predefined frameworks and libraries makes it easier to create "web sites and software". Fortunately for us (developers) at the same time user demands grow higher and higher. Web-Sites 'need' to become more dynamic, styled, responsive, ... simple CRUD 'needs' to turn into a 3-trier distributed middleware architecture.

tl;dr software complexity is a constant. Progress in easing development is compensated by increasing demands.

Re: Why do web sites and software take so long to build? And why is it so hard?

#37

Post by apl »

swecker wrote: post #34

I think of software development more as an art than a science. If those questions were asked of a professional painter I think we would hear similar responses. It's all by hand, colors shapes and textures can be amazingly complex in their combinations, standards would only inhibit a true artist, and a painter usually has no idea at all when a masterpiece might come forth. So my answer to "Why is is so hard?" would be…

That's outrageous self-glorification and utterly misleading. Software development qualifies as much as an art as other forms of engineering and most crafts do -- somewhat, at the highest levels, but generally not. Don't justify the shortcomings of current software engineering by nebulously labeling it a form of art.

Re: Why do web sites and software take so long to build? And why is it so hard?

#38

Post by Swizec »

Building software is often compared to building houses. If civil engineers can design and create a very complex bridge or dam or whatever, and it keeps working for 150 years without breaking. Why can't us software engineers make software that will work without bugs for more than 10 minutes?

Problem is, the two aren't really comparable.

A while ago someone wrote an article that we are Software Gardeners not engineers[1]. And I think that comparison is much more apt.

Software is maleable, it can change, the layers of complexity build up so high it's impossible for any one person, or team of persons, to fully understand. Even the mathematical theory of software isn't fully understood yet, and yet we are piling those little lacks of understanding on every layer of the architecture ... they add up and things become monstrous.

BUT! This is a good thing. Software isn't a physical object. I want to be able to change the specs half way through a project rather than having to wait another 50 years to design a better bridge. This makes progress quicker.

As always, when progress is quick (and easy), it is also messy.

For example: it took builders thousands of years to invent The Arch and trully revolutionize the industry. It took software only a couple of decades to go from machine code to python.

Oh and never forget, the Turing Machine was "invented" in 1948. Our lives revolve around an industry the theoretic principles of which were first defined less than 70 years ago.

[1] http://chrisaitchison.com/2011/05/03/you-are-not-a-software-...

Re: Why do web sites and software take so long to build? And why is it so hard?

#39

Post by davedx »

ExpiredLink wrote: post #36

Is it? > meta name="generator" content="WordPress 3.0.1" Was it hard to create this blog entry? Obviously he has a point but, in general, the abundance of predefined frameworks and libraries makes it easier to create "web sites and software". Fortunately for us (developers) at the same time user demands grow higher and higher. Web-Sites 'need' to become more dynamic, styled, responsive, ... simple CRUD 'needs' to tur…

I think that's a tautology.

In my professional experience (I started programming 20 years ago and worked on my first 'enterprise' Java project 12 or so ago), software complexity has become much greater. If you look at just the different areas you need to be proficient in now to build 'enterprise' software compared to 20 years ago, it's insane.

What I think has happened is that experienced developers have constantly had to keep up with learning to account for this. I don't think complexity has stayed constant - we have expanded our skill set and knowledge to keep up.

Post reply on HN