Live data from Hacker News

Databases have failed the web

josephg.com

51–60 of 144 posts

Re: Databases have failed the web

#51

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The articles point is that there's a bit more to Stack Overflow than the four database servers it sits on, and because MS SQL is treated as a monolith,a stupid amount of development time, the world over, goes into doing things the database, arguably, could be configured to do itself. I click the accept answer button on my web browser, stack overflow stack reads the cookie my browser presented, does a bunch of logic i…

I prefer Ruby/Python/JavaScript/.Net to PL/SQL thank you very much

Re: Databases have failed the web

#52
Follow the money. The notion that databases have failed the web was put to Mike Stonebraker of Ingres fame at a seminar I attended back in the late 1990s. The actual question was this: "Why aren't database vendors building object oriented databases for the web?"

Object oriented databases store binary objects as opposed to object-relational databases Mike pioneered with Postgres. The plan was for CORBA, common object request broker architecture, to shuffle versioned objects around the web and object oriented databases would store binary code objects.

Mike's answer? "Because the money is in transaction processing. Banks pay millions of dollars for transaciton systems. Transactions are the meat-and-potatoes of database sales."

In some sense database companies are wise for ignoring the web. Web companies are now hell bent on "move fast and break things." A transaction database is the antithesis of move fast and break. Transactions move deliberately with consistency. Durability says I can unplug my database at any time and bring it back up. This is the opposite mind set of move fast and break things.

Here in the valley whole QA departments are being disbanded in favor of LEAN and move fast and break things. All of this is antithetical to the database world of transaction processing.

At they say: you get what you pay for. What the enterprise companies are paying for is exactly what is being delivered. Open Source fits the every changing, move fast and break things of the web world.

To whit, you might see database vendors move into the web space some day if in fact companies are willing to pay millions of dollars for them.

Re: Databases have failed the web

#53
post #18

Earlier quoted context omitted.

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

Maybe if there was better support at the connection layer, this might be better. If ActiveRecord (Rails) allowed you to pass a free-form ruby function and built a procedure and executed it in SQL, I'd be all for it, but I'm not writing a one-off procedure and mantaining it.

What is there to maintain? It's not hard to directly emit SQL to ActiveRecord

Re: Databases have failed the web

#55

I think the author failed to educate himself. Databases are a miracle product. If you think of an application as a car, the database is the engine. The idea that you have a platform that can do everything without the abstraction of a separate data storage/query platform, that exists too. I'd argue that FileMaker, Lisp, MUMPS, and a few others basically do this in different ways. I used to be a DBA at a company where…

^ This reply is very good, and is an example of "Systems Perspective."

Re: Databases have failed the web

#56

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

Exactly! How about "developers have failed the databases" The majority of the developers don't use (or don't know how to) views/stored procedures/roles/triggers. Whenever you mention something like this you hear "omg, you put business logic in the db? everybody knows that does not scale" or "we deconstruct our request into a few simple queries instead of a join so we run super optimal"

Re: Databases have failed the web

#57

It isn't that "everybody emulates the VT-100 for some reason" but that the VT-100 was the first terminal to support the ANSI standard for control codes and that that standard has been evolving ever since.

Yep, I also wanted to add that if you use Mac, or Unix you most likely don't use VT-100.

Yes, they all support VT-100, but that's because it became a standard, and essentially any device understands it. For example if you connect to a network switch you know that it will at least support VT-100 so you have a common protocol to communicate with it, and be able to switch to a modern protocol if the device supports it.

I feel like the author is a front end developer and just hates the fact that JavaScript in the browser can't communicate with a database server directly.

I don't understand though, if there is a need for something like that, and it is also possible that database could provide a generic enough interface that would work, then there must be an application that you install together with the database which would do exactly what he wants.

There's no need to add this functionality to a database, because it only adds a complexity that majority of people who use a database won't need.

Re: Databases have failed the web

#58
I haven't read all the comments here yet, but I'm going to throw this out anyway...

My confidence in the author took a hit at the point he called a PDP-11 a "mainframe". One would think a computer scientist would know what computers fall into what "generation". I guess "history of computation" is just not taught as part of such a degree anymore...?

But...I decided to read on, thinking maybe things would get better, and to give him the benefit of doubt. I think somewhere in there was maybe a few points to think about, but ultimately it almost looks like he has some weird problems with "separation of concerns", and maybe doesn't understand why that would be a bad thing for scaling...

...which again, I find odd for a computer scientist.

Furthermore, he seems to ignore the great amount of improvements and changes which have occurred in the database software/engine and server world; today's DBs and DB systems are -nothing- like they were back in the early 90s when I started my career (as a fresh high-school graduate). Yeah, we still used VT-100 terminals (then quickly transitioned to PCs - running VT-100 terminal emulators, of course), and things were starting to transition to PC apps communicating to the servers - and I am sure there were SQL injection issues (and no, we didn't think about that) - but things have, over the decades (yeesh - getting old here!) have changed for the better!

Could they be better? Certainly! Are there things the DB server could be doing to make life easier for the app? Yes (and some of this has been implemented - ie, when developed properly, your queries can be "sanitized" at the DB server level - but you know, you should still do this at the app and browser level too - just in case). Realtime updates and notifications? That's pretty much there as well - but ultimately, a lot still has to be done at other levels.

...and that's not a bad thing, imho.

Post reply on HN