Why do web sites and software take so long to build? And why is it so hard?

1. Language that doesn't require me to connect to the external system and use new brain-dead language to do such basic thing as persisting and efficiently accessing data. How do you declare persistent array of objects that you will be wanting to access by some of their properties in your favourite language without keeping the whole thing in RAM? I'd love to have my data stored in XML if I could query it without loading the whole file into memory and if I could hint it what XPaths I'll be accessing in the future so it can create some indexes to make those accesses faster.

2. Tool that I could use to convert photoshop design to sane subset of HTML and CSS and perhaps JavaScript without manually writing text documents all by hand. Dreamweaver is just set of buttons to click to insert tags and that's not what I mean. I think of something more like a GUI designer where I could load psd, or multiple psd-s and use them as a blueprint and source of bitmaps while creating a webpage. In MS Visual Studio GUI designer has a nice thing called Anchor. It allows you to anchor borders of your component at fixed distances from borders of it's parents. If people who defined HTML standards embraced that ideas then webdevelopment of recent 10 years would be flowers and bunnies instead of franticly searching how to make the browsers actually fill the height of the parent when you say "height: 100%" (hint: you can't).

3. Tool that could present me with graphical view of what components my webapp consists of and how they are connected. Something like database diagrams but for living breathing components. It's really tiring to making sense of what your app actually does just by looking through a small hole of the size of letters in the debugger and stack trace.

4. I'd like to see, while inspecting a method, all the places in my code where this method is surely called from and all the places this method might be called from (because of inheritance, function pointers, dynamic calls etc.). Function call is the connection between component. It's really nasty that when looking at source point of the connection then you can see the destination but when looking at destination you have no idea what are the source points.

What webdevelopment lacks in my opinion are just proper (visual) tools to deal with its complexity and inconveniences. Decade or two of this crap and you really are fed up because there are less and less interesting problems to solve and same drudgery over and over. Standard component interfaces would also be nice but unless you have a generally adopted tool that favours some standard there won't be any standard. Current state of the art web-development tool is text editor with syntax highlighting and auto-complete broken to various degrees. The only thing it favours is not keeping HTML, JavaScript and CSS inside strings because then it refuses to highlight it. It does the same thing about SQL but that really doesn't bother people enough because SQL is already lost cause.

It just takes time to make an entire industry work smoothly and software engineering is still a young discipline. When I look round at the innovation going on lately - the rise of alternate JVM languages, increased expressiveness in languages like Ruby, frameworks like Rails, Django, Bootstrap, Less & SASS, compile-to-JS projects like CoffeeScript & ClojureScript, renewed interest in visual programming ideas like LightTable and many more, it seems we've put our foot on the gas again. I'm pretty optimistic the programming tools of tomorrow will fix the day-to-day annoyances, leaving us more time to concentrate on the core problems.

It's not just that you'll run into one unique requirement; you might run into a unique combination of requirements, each of which already has proven solutions, but with no good way to glue it all together. That's the reason C programmers still sometimes write their own string handling or memory allocation code despite that stuff being literally in the standard library.

Writing general purpose software components is hard. If you're creating a product, you know what kind of component you need, and you don't really care about anyone else. If you're making a general purpose component, you have almost literally no possible way of even comprehending, much less fulfilling, all the requirements of every product that could potentially use your component.

You're always going to notice a difference between something that's been cobbled together out of spare parts and something that's been designed to fit an integral product vision. There's a reason we've been hearing about reusing program components for literally decades. I'm sure some chunk of the problem will be broken off and solved, some kind of standard solution to the CRUD app or something, but there are still going to be products out there that need real engineering, not just component assembly.