Databases have failed the web

I agree that more business logic needs to be "closer" to the DB - if not in the DB itself (using those facilities you mentioned), then at a level very nearby. There's a very good reason for this:

Showing a consistent interface for data update and query.

When there are multiple teams developing against the same DB server, they might be able to see most or all of the tables, but they each might implement updates/selects (SQL queries) that exhibit extremely different results - which either present or update the data in potentially incompatible or inconsistent ways.

This ultimately manifests itself as errors/bugs which can be very difficult to figure out what went wrong where and why (and correcting the data becomes a nightmare).

Things get even worse when you have people querying the data via ad-hoc methods (like ODBC to an Excel file or something weird like that). Essentially, it is possible (likely) that every team sees a slightly (or greatly) different view from each other.

Coupling the BL closer to the DB level would prevent this; whether via the tools mentioned (views, triggers, etc) or something else. Ultimately, app developers shouldn't be hitting the DB tables directly, but going through these interfaces.

Of course - that brings up a host of problems on its own (table updates become an issue of coordination and conflict between teams, etc) - and there's also the issue of implementing versioning of the business logic (and how that relates to current/past DB structure)...

...but implementing it would solve a lot of headaches (and create more I guess).

/sigh

In the mid-1990s, some of the most complex web applications were implemented inside the database! I was the technical lead for a production web app written in... 250,000 lines of PL/SQL. This model had some significant advantages and worked surprisingly well considering that Oracle was not designed with that (ab)use case in mind.

So why didn't this model become common given that it was relatively elegant and capable? A few reasons:

- It required developers to be sophisticated at both using the database and creating web front-ends, since they were inextricably mixed. Even today, most developers are strictly one or the other, not both.

- The tooling inside the database was not designed for this use case, so while the architectural model was elegant, the development environment was piggybacking on functionality designed for reporting systems to drive interactive websites. This got better with time but by then no one cared. For a minimal website, hacking together a couple Perl scripts had a lower learning curve but was less capable.

- At the time, only a couple databases had the level of sophistication and features to make this feasible. Like Oracle. The upfront licensing costs were outrageously high, so there was no cheap way to bootstrap or incrementally grow into your application.

A 2017 version of this would work very well, database engines have much more sophisticated capabilities than back then that would make the development and operations experience pretty efficient and nice. As a practical matter no one designs web apps this way any more so there is no market for it.

Your argument may hold for Oracle and IBM, but the web crowd produced MySQL and Postgres, investing effectively millions of dollars of developer salaries and sweat equity, and those systems have evolved more towards features already in the commercial RDBMSes than towards a new, web-oriented DB flavor. Apparently web backend developers said "we want more of Oracles features" and not "we want the DB to handle these web-specific issues". At least, not until NoSQL systems like Mongo came around.

Post author here.

> Databases are a miracle product. If you think of an application as a car, the database is the engine.

Yeah - you're totally right. Thats kind of why I'm complaining about them - because we use them so much I think the potential benefits from improving more them are massive.

The modern web stack is a mess, and changes all the time. But there's a handful of features we need in just about every application - login, update feeds, data-based rendering (computed views), etc. But stored procs aren't good enough - they're inaccessible, hard to debug and usually require their own (non-standard, barely known) language to develop in. So lots of developers write functions outside of the database through multiple queries, and in doing so will often break the database's transactional consistency model.

Databases have managed to stick around with us throughout the evolution of the web. I'm being critical, but I'm doing it out of love. I'm saying that databases have the potential to provide so much more to the modern web ecosystem than just data storage. They could do access control, and run stored proc-equivalent functions or computed views from whatever languages my app is written in. If I need some complicated data flow where data needs to be updated in realtime in a secondary db (like elasticsearch), that should be dead simple to do too. I didn't get into it in that article but I believe we should be able to extend our databases to share data flow logic between applications. Or just build a lot of this stuff into databases so its reusable between projects.

I suspect it's because many developers are introduced to programming through web development. The pedagogy fails them and they go on to successful careers believing falsehoods.

It's hard enough to learn a programming language the first time that I can see how ORMs and frameworks relieve the burden of having to learn SQL, RDBMSs, etc.

However it's a convenient lie that seems to rarely ever be corrected.

The reality is that ORMs are exactly the wrong abstraction. They introduce premature pessimization into your application. They are not structured around how your data should be designed but how data-structures in your language of choice are designed. And left uncorrected they lead many developers to the mistaken belief that RDBMSs are slow, inflexible, etc, etc.

It turns out that relational algebra gives us great tools for querying complex data structures and relationships and building consistent, correct abstractions.

RDBMS servers have great facilities for managing and interacting with our data. Procedural languages, authentication, authorization, inter-process communication, etc, etc.