Live data from Hacker News

Ask HN: What could a modern database do that PostgreSQL and MySQL can't

news.ycombinator.com

231–240 of 326 posts

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#231
post #212

1. Streaming SQL Real-time queries should be the default. Some kind of smart query invalidation (similar to incremental view maintenance) as a result of DML statements. 2. Can run in a web browser / mobile phone Your client-side cache is usually a messy denormalization of your SQL database. For offline/local-first apps you essentially need to run your entire backend API in your browser. So you should be running an SQ…

A lot of the points you mention are really interesting to me, as I've been coming to similar conclusions recently. What are good choices that solve these particularly in the context of js/clientside apps?

absurd-sql[1] is the coolest thing I've seen in this space so far. sqlite.js brings sqlite to the browser via wasm, and then absurd-sql implements the storage layer in IndexedDb. So now we have an sqlite compatible db in the browser to play with.

There is also alasql[3] which is implemented in js, and lovefield[2] from Google which seems like an experiment that is now abandoned.

First, you could implement a REST/GraphQL cache in SQL. This would require maintaining mappings of your API response fields to SQL tables.

Going further, you could implement your backend API on the client, and have requests to it go directly to your browser SQL db. The benefit of this is you write your API once, and you get full offline support. If you don't need "local-first" then it's just a matter of figuring out when a query becomes invalid on the server. I could show instant results from local db, and then send a query to the server to ask if the data I already have in my cache can fulfill this request. Could optimize this on the server by listening to DML queries via WAL/LISTEN. WebSockets would be used to maintain real-time updates to only the data you are viewing.

You could also just use SQL directly as your UI API and then trigger these queries on the backend (respecting security of course).

What's doesn't feel optimal though is subscribing to sqlite.js updates in the browser. This makes me feel like we need an SQL db written in JS.

Also, if our DB is running in the same execution environment as our frontend and has one consumer, we could store each row as a JS object, and then reference this directly in our views, and subscribe to updates on a row-by-row basis. So if you are rendering a large table, if a column in a single row changes, when this row is updated in the database, instead of re-rendering the entire table manually, (or smartly detecting if the query was invalidated), you just bind to updates on the row instance, which is what the db is actually storing. I think this would reduce huge amounts of code.

Local-first is a little more difficult. There is some good exploration here[4].

I think writing a db in JS on the backend is not such a bad idea either. The hot paths would be written as Rust addons, but the majority of stuff people want today is just implementing the right logic to automatically handle denormalizations and subscriptions which we already hack together in our application layer using JS et al.

    [1]: https://github.com/jlongster/absurd-sql
    [2]: https://github.com/google/lovefield
    [3]: https://github.com/agershun/alasql
    [4]: https://actualbudget.com/blog/

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#232
post #229

Earlier quoted context omitted.

> Zero impedance mismatch between database and application representation Does this include informacion hiding/encapsulation? (to prevent saved objects' internal representation from being exposed). Traditional databases don't have an encapsulation mechanism AFAIK, which is one of the reasons for impedance mismatch. This is important because it is a good practice for client code to make no assumptions about the intern…

> Traditional databases don't have an encapsulation mechanism AFAIK, which is one of the reasons for impedance mismatch. It actually does, those are views and functions. The real problem with impedance mismatch is that SQL is declarative (you say what you want and database figures out how to get it) when most programming languages are iterative (you say what should be done). The issue is that you have two very differ…

> For example with NoSQL you need to know how the data will be used so you correctly plan how it will be stored. If application changes you might need to restructure the entire data.

This point is very important, and well explained in Stonebraker's paper "What Goes Around Comes Around". What is most interesting is that he is actually talking about half a century old pre-relational IMS IBM databases, but they had exactly the same issue, hence the paper's title. Codd invented the relational model after watching how developers struggled with the very problem you mentioned.

Stonebraker famously quipped that "NoSQL really stands for not-yet-SQL".

He also addresses the impedance matching issue in the "OO databases" section; there is actually a lot more to it, and he gives it all an insider's historical perspective.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#233

Earlier quoted context omitted.

The Superbase ( https://github.com/supabase/realtime ) approach is really interesting. It listens to the logical replication stream. Makes a lot of sense to me. Unfortunately our postgres instances hosted on heroku don't expose this, so I've been unable to try it out.

Hasura discusses why they chose polling here ( https://github.com/hasura/graphql-engine/blob/master/archite... ). > WAL: Reliable stream, but LR slots are expensive which makes horizontal scaling hard, and are often not available on managed database vendors. Heavy write loads can pollute the WAL and will need throttling at the application layer. Would like to hear Supabase's response to this.

Hasura's approach is great and there are just tradeoffs between approaches.

Sure, slots (can be) expensive; GCP doesn't offer slots but all the others do as far as I know (and I see that as a GCP issue, not a Supabase issue); a lot of writes can definitely fill the WAL.

Stepping back, the main comment I can give is that we each approach problems with a slightly different philosophy. Hasura tackles problems using middleware and Supabase tackles problems using the database. Both are fine.

Our approach is important (to us) as we evolve the product. For example, we are adding Row Level Security to our Realtime instance. Because it all sits in the database it's essentially just a Postgres extension, minimizing chatter between a middleware server and the database. Also because it's at the "bottom of the stack", other vendors/integrations can make use of the functionality we provide.

(supabase cofounder)

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#234
post #184

- Blurring (safely!) the line between database and the app using it: transparently switch between bringing data to compute, or compute to data. - Comprehensive auto tuning: automatic index creation, automatic schema tuning, dynamically switching between column/row-oriented, etc. User specifies SLOs, database does the rest. - Deeply related to the previous two points: perfect horizontal scalability - Configurable per-…

> Blurring (safely!) the line between database and the app using it

What do you mean by this?

> Comprehensive auto tuning...automatic schema tuning

You should be able to maintain a logical database schema, and then flip a toggle for things you want to have denormalized (and eventually have the db just do it automatically). Or maybe even take a blob of unstructured data and automatically normalize it.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#235

Earlier quoted context omitted.

It always rubs me the wrong way - all those paxos/raft approaches (which are great, but...) simply elect the leader to pick writes. In that sense there is no distribution of computation at all. It's still single target that has to cruch through updates. Replication is just for reads. Are we going to have something better anytime soon? Like real distribution, when you add more servers writes distribute as well?

> Like real distribution, when you add more servers writes distribute as well? There's a really interesting suggestion in The Mythical Man Month. He suggests that instead of hiring more programmers to work in parallel, maybe we should scale teams by keeping one person writing all the code but have a whole team supporting them. I don't know how well that works for programming, but with databases I think its a great id…

>There's a really interesting suggestion in The Mythical Man Month. He suggests that instead of hiring more programmers to work in parallel, maybe we should scale teams by keeping one person writing all the code but have a whole team supporting them.

Sounds like mob programming!

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#236

It's always been insane to me that databases aren't better at migrations. There are so many blog posts out there on how to do zero downtime migrations, and I feel like a lot more of that could probably be abstracted away.

There are so many times I wish a blog post was an app that just had a nice gui to do guide me through the steps they suggest, instead of manually modifying config files and running bash scripts. SQL GUI tooling is lacking.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#237
CouchDB.

- Very smart clustering/redundancy features

- Can be queried over HTTP

- Has a great API

- Counterpart project (Pouch DB) can work on the client

- Queries can be cached (it's HTTP)

- Said to be 'schemaless' but can be structured

The project is always overlooked, but I still feel it's one of the better NoSQL datastores.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#238

Lots of talk of CockroachDB in this thread but no mention of vitess.io? Shame. While I don’t care for MySQL the folks at planetscaleDB (hosted vitess) are doing amazing work. So are the CRDB folks.

Because HN is predominantly a Postgres and Anti Oracle community, so the use of MySQL doesn't fit their ideology. Which is unfortunate because I think Vitess and Planetscale is quite nice.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#239
post #168

Earlier quoted context omitted.

You might be surprised to learn how common the "store the results of the user's query into a temporary table" pattern is.

Temporary tables in cockroach exist, but the implementation was done largely to fulfill compatibility rather than for serious use. The implementation effectively just creates real tables that get cleaned up; they have all the same durability and distributed state despite not being accessible outside of the current session. Getting something done here turned out to be a big deal in order to get ORM and driver tests to…

Thanks for all of that information in those 2 posts.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#240
post #58

My biggest problem with databases is always versioning. IE renaming a column will break old clients. If there was a way you could have multiple schema versions so you could upgrade database then clients later it would be the best. EDIT: yes thanks for the comments, views and creating and API layers and adding instead of subtracting do all work, but I believe they're all workarounds for the underlying problem. Fixing…

Becomes too complex. You have to maintain mapping of old columns to new columns and when you are splitting tables it becomes too crazy.
Post reply on HN