Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases have failed the web

josephg.com

Post reply on HN
Print view View original

61–70 of 144 posts

Re: Databases have failed the web

#61

Post by geon »

daliwali wrote: post #29
dpark wrote: post #18

Earlier quoted context omitted.

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

Many DBs weren't built to be exposed on the open web. The web didn't even exist at the time some popular databases were invented, and even after the fact, it's still not the purpose. I would even argue that DBs have overstepped their bounds already with what programming abilities some have built-in. They should be focused on reading and writing structured data efficiently and correctly, not much more than that. Which…

> DBs have overstepped their bounds

And they have a stringly typed interface! It sucks! When both the languages inside and outside the database are typed, why should the interface between them not be?

It is just unfathomably bad.

Re: Databases have failed the web

#62

Post by geon »

dom0 wrote: post #16
pbnjay wrote: post #8

I really disagree with this - separation of concerns is incredibly important. Why does the database server need to do everything my application server does now? I don't want to have to know about how my database works internally just to implement a new feature in my application. I don't want to worry about a junior dev corrupting data while building a login page. My "simple old-school" database is reliable and consis…

Fully agreed. The observation that SQL still works as well as it does today as it did in the 70s suggests that it's a good and valueable abstraction.

> SQL still works as well as it does today

Or... It sucks as badly now as it did back then.

Re: Databases have failed the web

#63

Post by Animats »

And, to fix it, he says, we need functional programming at the database level! What?

Today's databases are probably the best part of the server side stack. The parts that talk HTTP, JSON, and do business logic are usually worse.

Re: Databases have failed the web

#64

Post by taeric »

fragmede wrote: post #39
okket wrote: post #30

Earlier quoted context omitted.

> DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. This is sane and good design. The db auth layer is for db management, not frontend auth (are you seriously advocating this?). > DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. This is just (initial) design / programmer laziness. I bet if you start to sca…

> The db auth layer is for db management, not frontend auth Yes, but why ? The database already has a table of database users, and the different permissions they're allowed. (Well, hopefully your webapp doesn't just do everything as the root DB user with * privs.) Is there that much to be gained from the database having a second table of users that you've created, and then you get to reimplement a bunch of logic in y…

You gain making users of your system a first class and distinct part of your application from those that maintain it.

Or, would you rather swap a regular user entry into and away from a maintenance role when you hire/fire someone? (Certainly doable, but seems extreme.)

You also gain the ability to severely lock down destructive permissions to the database behind fairly rigid authentication rules.

Re: Databases have failed the web

#65

Post by sk5t »

ruslan_talpa wrote: post #56
agentultra wrote: post #3

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

Exactly! How about "developers have failed the databases" The majority of the developers don't use (or don't know how to) views/stored procedures/roles/triggers. Whenever you mention something like this you hear "omg, you put business logic in the db? everybody knows that does not scale" or "we deconstruct our request into a few simple queries instead of a join so we run super optimal"

> deconstruct our request into a few simple queries

Makes sense if you can cache only portions of the result, and/or the query optimizer or database layout actually struggle with the join.

Re: Databases have failed the web

#66

Post by mixmastamyk »

moron4hire wrote: post #47
fragmede wrote: post #39

Earlier quoted context omitted.

> The db auth layer is for db management, not frontend auth Yes, but why ? The database already has a table of database users, and the different permissions they're allowed. (Well, hopefully your webapp doesn't just do everything as the root DB user with * privs.) Is there that much to be gained from the database having a second table of users that you've created, and then you get to reimplement a bunch of logic in y…

Because traditionally the database was managed by IT, not by engineering. IT came up with the idea that it should take forever to approve changes to the database, so engineering invented ways around their can-don't attitude. Can you imagine any CTO approving a web-based signup page being allowed to change the users to the database?!

If it only adds restricted users is that a big problem?

Re: Databases have failed the web

#67

Post by YCode »

hive_mind wrote: post #44
dom0 wrote: post #16

Earlier quoted context omitted.

Fully agreed. The observation that SQL still works as well as it does today as it did in the 70s suggests that it's a good and valueable abstraction.

Lindy Effect.

I can't tell if you're describing or disputing the comment you replied to.

Are you saying SQL survived only because people have used it for so long and therefore its sustained use isn't an indicator of value?

Re: Databases have failed the web

#68

Post by moron4hire »

mixmastamyk wrote: post #66
moron4hire wrote: post #47

Earlier quoted context omitted.

Because traditionally the database was managed by IT, not by engineering. IT came up with the idea that it should take forever to approve changes to the database, so engineering invented ways around their can-don't attitude. Can you imagine any CTO approving a web-based signup page being allowed to change the users to the database?!

If it only adds restricted users is that a big problem?

The problem is not an issue of technology.

Re: Databases have failed the web

#69

Post by josephg »

ericHosick wrote: post #34

> All because we're programming against a frozen database spec. Relational databases, unlike XML, JSON, Key/Value stores and ORMs, do not pre-suppose document structure. On top of that, it is very easy to create new relations (entities) using Views. On top of that, you get a real algebra to play with: relational algebra. SQL makes it crazy easy, in real time, to see your data in any hierarchal manner you like (via de…

Hi, post author here.

My point about the frozen spec doesn't have anything to do with the data format. Its that modern databases don't expose the API that modern web applications actually want, like:

- Integrated access control

- Realtime updates (sorta, kinda)

- Resumable event logs

- The ability to do computed views in my language (eg, make a computed view using a nodejs react static rendering process)

Because databases lack these features, modern web developers often end up rewriting them at the application level, for every app.

Re: Databases have failed the web

#70

Post by dasil003 »

geon wrote: post #62
dom0 wrote: post #16

Earlier quoted context omitted.

Fully agreed. The observation that SQL still works as well as it does today as it did in the 70s suggests that it's a good and valueable abstraction.

> SQL still works as well as it does today Or... It sucks as badly now as it did back then.

Well, no. There were a zillion databases competing and SQL is what survived. The NoSQL movement has created a bunch of useful DBs for specific scaling workloads, and of course there are graph DBs and other special purpose DBs, but SQL still dominates, and with good reason: it's the right balance of performance, flexibility, structure and integrity that makes it an excellent hedge for the early phases of application development when you're not yet sure what exactly the workload is going to look like.
Post reply on HN