Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What could a modern database do that PostgreSQL and MySQL can't

news.ycombinator.com

Post reply on HN
Print view View original

291–300 of 326 posts

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#291

Post by pmelendez »

yarcob wrote: post #261
GeneralMayhem wrote: post #244

Lots of people are citing cutting-edge bells and whistles for advanced query plans and data types, but after working at $BIGCO and seeing what exists there (and has existed for years), I get frustrated by the lack of ergonomic and operational essentials in the public-cloud options. 1. Horizontal scaling. PG can do sharding or replication, but that's not the same thing. Manual resharding is something that, once you've…

> Horizontal scaling I think the importance of horizontal scaling is overhyped. 99% of PostgreSQL applications are at a size where a single machine can easily handle the workload. To enable horizontal scaling, you need to make so many tradeoffs that I don't think it's worth it for most applications.

I see people responding to this thread with examples of high-volume workflows that use a small number of machines for the database layer.

I think those examples are missing the point. One of the examples uses a Redis cluster, which is a sign that the DB alone can't support the workflow.

Another example is using a database for certificate issuance. Which I suspect is using one table or multiple that can be shard and it doesn't suffer from lock contention or joints of multiple tables.

I wouldn't be brave enough to count the number of cases where horizontal scaling is needed but I would say it is definitely not zero, especially for read replicas when high availability is needed

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#292

Post by Bubblegum21 »

GeneralMayhem wrote: post #244

Lots of people are citing cutting-edge bells and whistles for advanced query plans and data types, but after working at $BIGCO and seeing what exists there (and has existed for years), I get frustrated by the lack of ergonomic and operational essentials in the public-cloud options. 1. Horizontal scaling. PG can do sharding or replication, but that's not the same thing. Manual resharding is something that, once you've…

Wanted to ask you more about the experiences you wrote. Have some similar thoughts as your later points. Do you have a contact method like an email address or something?

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#293

Post by calmoo »

pas wrote: post #133
calmoo wrote: post #113

Earlier quoted context omitted.

CAP theorem. Choose two, it's a fundamental limitation of scaling a database.

yes, but no. because in reality what we really want is not true pure AP or CP (or CA, which you can't have anyway) https://www.youtube.com/watch?v=hUd_9FENShA (CAP is a very important result about a very strong assumption of consistency: linearizable events, but for that you can't lose any messages [if I remember it correctly], otherwise the system will become inconsistent)

Thanks for the video, very interesting.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#294

Post by LoriP »

KingOfCoders wrote: post #266
lazy_kangaroo wrote: post #263

Earlier quoted context omitted.

where did you get the 99% metric ? Most of the companies even with a single saas product have insane amount of data these days. Not just application data, There is also a whole lot of analytical data collected at every step of the product usage cycle.

From managing databases over the last two decades (starting with Ingres/mSQL), hardware has growing much faster (RAM/CPU/NVMe) than the data needs. I remember the first time we put 2Tb RAM into a machine to handle most analytics in memory. From my experience TimescaleDB is fast and takes a lot of data in. If you're multi tenant, it's usually easy to shard. And we do dataware housing on BigQuery, no need to have your…

TimescaleDB also offers a good lot of supplementary functions to the PostgeSQL core product to help with time series data analysis... saves a lot of SQL acrobatics!

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#295

Post by eurasiantiger »

Glyptodon wrote: post #210

This is pretty stupid, but I'd be happy to see a SQL dialect where a statement can start with a FROM clause. This is also pretty stupid, but there are also some cases where I wouldn't hate some sort of "JOIN AS CHILD" feature along with results that aren't 100% tabular.

You’re looking for a graph database.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#296

Post by remram »

Nican wrote: post #46
mirekrusin wrote: post #33

Earlier quoted context omitted.

It always rubs me the wrong way - all those paxos/raft approaches (which are great, but...) simply elect the leader to pick writes. In that sense there is no distribution of computation at all. It's still single target that has to cruch through updates. Replication is just for reads. Are we going to have something better anytime soon? Like real distribution, when you add more servers writes distribute as well?

There are two ways I see databases doing paxos. The basic way, like some databases like Percona, basically is a single raft across the whole database. It can help with high availability, but the database scale is still kind of constrained to the capability of a single writer. What you really want is databases like CRDB/Yugabyte/TiDB, which are sharding+raft. Tables are sharded into 128MB chunks, and each chunk has th…

Doesn't CockroachDB work this way as well, with each "range" running a separate Raft? That is what I get from https://www.cockroachlabs.com/docs/stable/architecture/overv...

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#297

Post by splike »

strken wrote: post #172
estebarb wrote: post #128

Materialize ( https://materialize.com/ ) is capable of performing SQL operations over a stream, using incremental calculation based on differential dataflow. CockroachDB is a distributed SQL database, with strong consistency. I think that Yugabyte is similar. Support for realtime changes, including queries over those changes, is better in other databases, like RethinkDB, OVSDB or Firebase. Relational is not always th…

I really want to do streaming queries over updates and inserts in a normal SQL database. Imagine writing SUBSCRIBE SELECT customers.id, customers.name FROM EVENTS(customer_purchases) AS cpe LEFT OUTER JOIN customers ON cpe.row.customer_id = customers.id WHERE cpe.row.product_id = '123abc' AND cpe.type IN ('update', 'insert') in postgres itself and just getting every incoming purchase for a particular product.

Someone with more in depth knowledge can tell me why this isn't a good idea, but I'm pretty sure you can get something very close to this with graphql subscription query against hasura backed with postgres

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#298

Post by Scarbutt »

takeda wrote: post #229
charlysl wrote: post #145

Earlier quoted context omitted.

> Zero impedance mismatch between database and application representation Does this include informacion hiding/encapsulation? (to prevent saved objects' internal representation from being exposed). Traditional databases don't have an encapsulation mechanism AFAIK, which is one of the reasons for impedance mismatch. This is important because it is a good practice for client code to make no assumptions about the intern…

> Traditional databases don't have an encapsulation mechanism AFAIK, which is one of the reasons for impedance mismatch. It actually does, those are views and functions. The real problem with impedance mismatch is that SQL is declarative (you say what you want and database figures out how to get it) when most programming languages are iterative (you say what should be done). The issue is that you have two very differ…

Your view of what impedance mismatch is doesn't sound accurate. It's not about declarative or imperative or syntax or strings, etc...

It's about data modeling, one models data using relations, the other models data in a hierarchical way (using maps, arrays, objects, etc...). They are two different ways to structure your data, hence the impedance mismatch.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#299

Post by zinclozenge »

eitland wrote: post #207
zinclozenge wrote: post #22

1. Automatic backups to an S3 compatible storage, out of the box. 2. Progressive automatic scalability. As load increases or storage runs out, the DB should be able to automatically. NewSQL databases do this already. 3. Tiered storage. 4. Support streaming, stream processing, in memory data structures, etc. I feel like this is one of those weird things but I keep wishing this were possible when I work on side project…

> 3. Tiered storage. Do you mean tablespaces? https://www.postgresql.org/docs/10/manage-ag-tablespaces.htm... They'll allow you to have some parts on database on faster devices for instance.

That does seem neat, but I was actually thinking more like off-loading cold data to S3, or some such. Maybe even having distinctions between (ultra) hot, warm, cold data.

Re: Ask HN: What could a modern database do that PostgreSQL and MySQL can't

#300

Post by zinclozenge »

tlack wrote: post #101
zinclozenge wrote: post #22

1. Automatic backups to an S3 compatible storage, out of the box. 2. Progressive automatic scalability. As load increases or storage runs out, the DB should be able to automatically. NewSQL databases do this already. 3. Tiered storage. 4. Support streaming, stream processing, in memory data structures, etc. I feel like this is one of those weird things but I keep wishing this were possible when I work on side project…

Great list. For #4, you should take a look at Erlang or Elixir - they have "just enough" concurrency with streaming primitives and a functional style that makes it easy to use. They make a lot of "bolt on" stuff you need for a regular startup (Redis, Celery, etc) superfluous.

I am actually super familiar with Erlang and Elixir and I agree with your assessment, but with the caveat that if you go that route you end up with your own hand-rolled solution that won't be wire/API compatible with the most used OS projects. Or in the case of Erlang/Elixir you'll be running it in process! But it is great for that reason.
Post reply on HN