Live data from Hacker News

Databases have failed the web

josephg.com

21–30 of 144 posts

Re: Databases have failed the web

#22
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

Maybe if there was better support at the connection layer, this might be better.

If ActiveRecord (Rails) allowed you to pass a free-form ruby function and built a procedure and executed it in SQL, I'd be all for it, but I'm not writing a one-off procedure and mantaining it.

Re: Databases have failed the web

#23
post #8

I really disagree with this - separation of concerns is incredibly important. Why does the database server need to do everything my application server does now? I don't want to have to know about how my database works internally just to implement a new feature in my application. I don't want to worry about a junior dev corrupting data while building a login page. My "simple old-school" database is reliable and consis…

Came here to basically say this much. Part of me feels already that databases are already too hugely complex. The "do one thing, well" part of me wants one application for data storage, and then middleware for relationship enforcement, stored procedures, etc. That's likely too pedantic for most but I really see wanting your data layer to be your application layer as a silly throwback.

Also isn't this a similar idea to what FileMaker Bento did?

Re: Databases have failed the web

#24
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

The PostgrREST project is an interesting approach so fill this gap:

https://postgrest.com/

Re: Databases have failed the web

#25
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

It sounds like you would like to look at something like postgrest. See https://github.com/begriffs/postgrest

Re: Databases have failed the web

#26
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

[deleted]

Re: Databases have failed the web

#27
I don't really understand his point. He says that SQL has been around for 40+ years, as if that's a bad thing, and doesn't really say anything else.

I guess he is complaining about the idea that the user's input doesn't get insert directly into the database, i.e., the connection would be browser->db, and instead we need an application component that reads from the browser and shapes to the DB.

At this point, we must ask if the author has ever worked with one of the thousands of applications that implements their "API" through stored functions. I won't condemn this method wholesale, but people have mostly moved on to more flexible representations for good reason.

The whole article seems predicated on a belief that most work should be on the consumer workstation, which is why he starts by talking about how much better it is that people now have desktop computers instead of clients that connect to a mainframe in the basement.

The author apparently doesn't grasp that the web acts just like those old terminal clients he refers to in beginning of the article. There's a big server running the application in someone's datacenter, and your browser is a thin client over the top of it, an interface into its inner workings.

He blames this on databases. (???) The browser doesn't provide the mechanisms to directly connect to arbitrary protocols.

I don't really think there's much substance here.

Re: Databases have failed the web

#29
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

Many DBs weren't built to be exposed on the open web. The web didn't even exist at the time some popular databases were invented, and even after the fact, it's still not the purpose.

I would even argue that DBs have overstepped their bounds already with what programming abilities some have built-in. They should be focused on reading and writing structured data efficiently and correctly, not much more than that. Which database to use should depend on access patterns and structural requirements. Let the application logic be handled at a higher layer instead of mixing responsibilities everywhere.

Re: Databases have failed the web

#30
post #18

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single database server. If our workloads require it we're prepared to scale out horizontally. I'm really looking forward to PostgreSQL 10's new parallel query features for some of our analytics work. Stack Overflow runs everything across what, 4 MS SQL Servers in total?…

The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relation…

> DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer.

This is sane and good design. The db auth layer is for db management, not frontend auth (are you seriously advocating this?).

> DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer.

This is just (initial) design / programmer laziness. I bet if you start to scale up, you will think a lot about moving your business logic further to the real data for safety and for performance reasons. This is the point when you realise that db systems are very different and switching from one to another is not that easy...

Post reply on HN