Why do web sites and software take so long to build? And why is it so hard? • Hacker News Board

Re: Why do web sites and software take so long to build? And why is it so hard?

#131

Post by [deleted] » Mon, May 07, 2012, 7:28 PM UTC

rrreese wrote: post #61

ExpiredLink wrote: post #42

Earlier quoted context omitted.

> I think in my ~20year career, I've had only 3 projects that were specified well enough up front that we just "built it according to the plans" and had a satisfied customer at the end. Overwhelmingy, some (or most) of the design gets "made up as we go along". Even on projects with several small forests worth of up-front documentation, there's almost always large areas of vagueness or outright contradictory requireme…

Right, I've worked on many projects where it wasn't really possible to gather all requirements up front. This is hugely frustrating to everyone involved but the fact remains that until you have at the very least a working prototype you will often not be able to generate a final requirements document. On the other hand when requirements are nailed from the very start, the development proccess has been swift, well exec…

[deleted]

Re: Why do web sites and software take so long to build? And why is it so hard?

#132

Post by fridek » Mon, May 07, 2012, 8:02 PM UTC

joshcrews wrote: post #100

fridek wrote: post #54

Earlier quoted context omitted.

I was working as a freelancer for a long time and I had a quite simple solution for lack of documentation. Before the first meeting I'd send a customer 12-page form to fill which covered most of the "what and why" of the project. It quickly turned out that most of people have no idea whatsoever about the product they wanted to buy. I know that it's partly our job to guide people through rough path, but if all custome…

The projects I do have never had final documentation before development started, and its not necessary. I got this process from another Ruby consultant and it works great - Get wireframes of main page views (usually there are about 6 of these. You can help them create this - Get them to write out, or describe the requirements and take notes - Break the wireframes and requirements into user stories and load it into pi…

That sounds like a good idea. Maybe not the best for me, as I always had trouble with drawing a line "our contract does not cover that" and I'd probably lose negotiations about how many points something is worth. It's still better than having no safe margin at all.

Re: Why do web sites and software take so long to build? And why is it so hard?

#133

Post by dhimes » Mon, May 07, 2012, 8:14 PM UTC

jarrett wrote: post #126

dhimes wrote: post #93

Earlier quoted context omitted.

We have regulations here in the US also (I had my house rebuilt 10 years ago and recently turned an outdoor deck into an indoor office). But our regulations are primarily safety-related. The wiring must be safe, the pipes plumbed so that drains drain and vents vent, and so on. They don't really inspect the quality of the craftsmanship. If your wall is not load-bearing, they care very little at all about how it is bui…

In the US, the regulations often do little to protect construction clients, particularly individual homeowners. When contractors fail to observe the code, what recourse does the client have? Most often, the only thing available would be a lawsuit. Lawsuits are always expensive. This is especially true when they hinge on a technical question such as: "Did the defendant observe [insert building code citation] while ins…

I agree that savvy rogue contractors can skirt the law and build crappy, unsafe stuff (like came to light in Miami FL after hurricane....Hugo, I think...). But what regulations do well is make sure the honest guys know how to be safe. Whether they go to their conferences and learn best practices is optional, and they have to decide if they want to bear the expense.

If they ignorantly (not maliciously) wire the bathroom in an unsafe manner, however, they will fail inspection and be made to fix it properly. This is useful to the individual homeowners. In a way it's similar to the adage that "locks keep the honest people from breaking in." With both regulations and locks, they are less likely to stop the criminals.

Re: Why do web sites and software take so long to build? And why is it so hard?

#134

Post by fridek » Mon, May 07, 2012, 8:14 PM UTC

eric-hu wrote: post #66

fridek wrote: post #54

Earlier quoted context omitted.

I was working as a freelancer for a long time and I had a quite simple solution for lack of documentation. Before the first meeting I'd send a customer 12-page form to fill which covered most of the "what and why" of the project. It quickly turned out that most of people have no idea whatsoever about the product they wanted to buy. I know that it's partly our job to guide people through rough path, but if all custome…

I, too would like to see this form. It would provide a lot of guidance for my company. We're knee deep in the consequences of scope creep right now.

It is in Polish, so I guess it won't be very useful. I may translate it and post somewhere if I have some free time. For now, try googling "website brief template", it's how I made my first version, then improved it over time. It's basically a compilation of most of the questions I had to ask during my previous projects. If you like to look like a funny guy try adding some "would you like a pictures of your cat on the website". It releases some tension associated with such formal document (which I usually attach to signed contract, just in case), but be prepared to hear "yes, I actually would like to have cats on my website".

It's probably worth mentioning that this approach is correct only for small and medium projects, like things possible to deploy with Wordpress ninja in team.

This may be a good place to start: http://www.methodandclass.com/article/write-a-web-site-brief...

Re: Why do web sites and software take so long to build? And why is it so hard?

#135

Post by kragen » Mon, May 07, 2012, 9:44 PM UTC

bigiain wrote: post #47

michaelneale wrote: post #23

Earlier quoted context omitted.

Exactly - this article makes the very common (still) (1) mistake of thinking construction of software == writing code. Construction of software is compiler/interpreter/all the machinery that kicks in to take some text or instructions and makes it do something. To correct the analogy with building physical things - you could compare the architect designing a bridge (and arguing with engineers about materials and budge…

" … construction of software == writing code … " In one sense there _is_ this part of "constructing software", and _largely_ it can be done by the software equivalent of stereotypical "construction workers". (This is what a lot of people who've tried outsourcing to India are trying to do.) The problem is, while you can collect a pickup full of Mexicans who can lay bricks / hang sheet rock / tar roofs on most street c…

The grandparent comment explained clearly why this is. It's because the work that can be done by "codemonkeys" who will hang sheetrock if you give them good directions, but can't make architectural or structural decisions, is already being done by the compiler and other such software, so the only work that's left for humans is the stuff that involves making those decisions.

You could probably regulate "software construction" and mandate specific methodologies, but we can see already what the result would be: just look at the SEI CMM Level 5 Certified software development teams that already exist: Wipro, R Systems, and so on — technically inept companies that only exist to rip off clients who don't know any better.

Any country that mandates that kind of development for all software will be rapidly left behind by the countries that don't as software becomes an increasingly important part of the 21st-century economy. They'll still have human beings laying their literal bricks and tarring their literal roofs, while the rest of us are living in robot-built houses full of fountains and sculptures, or dynamically-reconfigurable programmable houses.

Re: Why do web sites and software take so long to build? And why is it so hard?

#136

Post by mattgreenrocks » Mon, May 07, 2012, 10:13 PM UTC

Czarnian wrote: post #127

rrreese wrote: post #61

Earlier quoted context omitted.

Right, I've worked on many projects where it wasn't really possible to gather all requirements up front. This is hugely frustrating to everyone involved but the fact remains that until you have at the very least a working prototype you will often not be able to generate a final requirements document. On the other hand when requirements are nailed from the very start, the development proccess has been swift, well exec…

Agreed. My experience is that there is way too much emphasis on languages, libraries, and frameworks. Nobody really talks about the methodologies of building software. For any given day on HN, how many articles are about how such-and-such extension for this-and-that language is the best thing since the great pyramids and how many are about how code review can help reduce bugs? Or how to more effectively gather requir…

Right on.

There is also a gap in discussion of software architecture. Everyone seems happy to use whatever their framework forces them into, or add more machinery to make up for it. Concepts like coupling and cohesion don't get much airtime, particularly with the prevalence of dynamic languages.

Re: Why do web sites and software take so long to build? And why is it so hard?

#137

Post by josephcooney » Mon, May 07, 2012, 10:15 PM UTC

kahawe wrote: post #89

trustfundbaby wrote: post #48

brilliant comment in the original post, just in case anybody missed it "When I had my bathroom remodelled, at first I thought it was organized amazingly well and why couldn’t software be like that? There was a designer from Expo, a primary contractor and subcontractors for tiling, painting, etc. a project workbook containing all the documents, including the design, and a logbook for every contractor to record their v…

To be fair, I can see one difference, though, at least here in Euroland: you have all sorts of standards and regulations that buildings and contractors have to adhere to and especially on bigger changes or constructions you would get an independent expert to check whether your contractors and subcontractors did obey all the rules and when they frakked up somewhere along the way, they have to fix it, typically on thei…

It's hard to regulate clear, timely and unambiguous communication.

Re: Why do web sites and software take so long to build? And why is it so hard?

#138

Post by apinstein » Mon, May 07, 2012, 10:17 PM UTC

Two comments:

1. When things don't line up like the plans in an analog project, you can just "line them up" and the original intent isn't seriously disturbed. That doesn't work in a hard-logic digital world.

2. That said, "real" projects still can come together more quickly because they are better about using interfaces and being loosely coupled than we (software developers) are. You can slightly move joists, light switches, etc, because the interfaces of how they interact with other components are better defined, and the glue code (ie wires, nails, cuts) can be trivially adjusted to make things fit and still get the desired end outputs.

#2 is partly why software "integration" projects can be hacky as hell and still work, they're just glue code for clearly defined interfaces. If we designed our internal applications with such simplicity and clearly-defined interfaces and wired them together rather than coupling them, it'd be more predictable.

Unfortunately that's really tough to do for software inventions, as there are so many new things where the interfaces might not even be able to be clearly defined.

Construction would look a lot more like software if you ordered wood but when the stack of wood showed up it was metal rods instead, or if it came pre-molded to the shape of another house.

There are just too many variables in de novo software. I used to have this debate with my old boss all the time. You either live with the reality and create new, valuable stuff, or if you want to build bridges, go write glue code for legacy systems.

Re: Why do web sites and software take so long to build? And why is it so hard?

#139

Post by eric-hu » Mon, May 07, 2012, 10:37 PM UTC

fridek wrote: post #134

eric-hu wrote: post #66

Earlier quoted context omitted.

I, too would like to see this form. It would provide a lot of guidance for my company. We're knee deep in the consequences of scope creep right now.

It is in Polish, so I guess it won't be very useful. I may translate it and post somewhere if I have some free time. For now, try googling "website brief template", it's how I made my first version, then improved it over time. It's basically a compilation of most of the questions I had to ask during my previous projects. If you like to look like a funny guy try adding some "would you like a pictures of your cat on th…

I think I'd be able to get the idea of your questions using Google translate--but the link helps too. Thanks!

Re: Why do web sites and software take so long to build? And why is it so hard?

#140

Post by charlieok » Mon, May 07, 2012, 10:39 PM UTC

I like the analogy of walking through a dark room with your hand on the wall. As long as your hand is on something you recognize, you know where you are in the room, what ground you've already covered, and you may have some idea of how you're going to reach the other side.

I started a rails project very meticulously. I had recently read through the 'Agile' book which closely tracks the development of the framework. I had a decent idea of what I was going to build and how.

It would be a tight ship, and code would be kept clean from start to finish. Before even starting the project, I invested a ridiculous amount of time and effort learning all the tools I would be using, evaluating every case where I had a choice between popular and well-supported tools. For each tool I chose, I thought about why using it was a good idea and how exactly it would fit into the workflow. I was determined that every commit would keep tests and documentation up to date with any code changes in that same commit.

All that is to say that I came into the project with carefully considered, but fairly rigid, opinions on most aspects of how to do the project. Perhaps not surprisingly, the other developers on the project did not share these opinions. They did things that ran against my idea of how to keep a project organized. Perhaps I also did things which ran against theirs.

Eventually, the state of disorganization reached a point that felt to me like letting go of a piece of furniture in a pitch-black room. I no longer had a sense of how the whole thing was put together. I no longer knew what all the 3rd party libraries we were using did, or why they were included in the project. I had planned to keep all libraries up to date but, for various reasons, we were falling behind in that regard and various roadblocks stood in the way of bringing things current. We had started using esoteric features of the database I didn't know very much about. I would do a deep dive of research to try and catch up my understanding, and then I'd fall farther behind in keeping my finger on the pulse of changes in the project. Soon, the codebase had dependencies on various servers in our organization that prevented me from simply running my own isolated instance of it.

I don't know a solution to this, yet. Right now I'm still trying to feel my way through that dark room.