Databases in 2021: A Year in Review • Hacker News Board

Re: Databases in 2021: A Year in Review

#91

Post by rapnie » Thu, Dec 30, 2021, 12:34 PM UTC

Nice collection of open source databases: https://codeberg.org/yarmo/delightful-databases

Re: Databases in 2021: A Year in Review

#92

Post by RedShift1 » Thu, Dec 30, 2021, 12:44 PM UTC

throwDec21 wrote: post #41

I'm just surprised that in 2021 BigQuery isn't more popular. I thought it would be top 10 by now, I moved to GCP because of it but feel like I'm the only one.

Bigquery seems like a tool for very large but static datasets. I also had a hard time figuring out the pricing so other than some test queries I moved on to other solutions.

Re: Databases in 2021: A Year in Review

#93

Post by RedShift1 » Thu, Dec 30, 2021, 12:45 PM UTC

cloudengineer94 wrote: post #77

Postgress is amazing, however I work with SAP HANA every day and I gotta say this thing is completely insane.

Well it is hard to beat keeping everything in RAM...

Re: Databases in 2021: A Year in Review

#94

Post by dreig » Thu, Dec 30, 2021, 1:18 PM UTC

rapnie wrote: post #91

Nice collection of open source databases: https://codeberg.org/yarmo/delightful-databases

There's a very nice and comprehensive database of databases https://dbdb.io/ started and maintained by Andy Pavlo and the CMU-DB group.

Re: Databases in 2021: A Year in Review

#95

Post by adamkl » Thu, Dec 30, 2021, 1:22 PM UTC

zffr wrote: post #29

The author is a professor at CMU who specializes in databases: https://www.cs.cmu.edu/~pavlo/ Not completely related, but his lectures on databases on YouTube are really good. Much better than the DB class I had at college.

Said lectures on YouTube: https://www.youtube.com/playlist?list=PLSE8ODhjZXjbohkNBWQs_...

A great way to learn more about the inner workings of databases, and entertaining too.

Another choice quote (from one of his lectures):

“There’s only two things I care about in life:

1. My wife

2. Databases

I don’t give a f#ck about anything else”

Re: Databases in 2021: A Year in Review

#96

Post by chishaku » Thu, Dec 30, 2021, 2:01 PM UTC

criticaltinker wrote: post #22

Databases in 2030: SQL DB finally succumbs to Graph DB as #1 Does anyone else feel like a caveman when modeling a many to many relationship in a normalized schema, and then querying via SQL? I’m surprised graph DBs aren’t more popular for this reason alone. Maybe it’s a far fetched dream, but perhaps a graph frontend can be slapped onto the Postgres backend.

Which db is that?

Either way, that’s not happening.

Re: Databases in 2021: A Year in Review

#97

Post by sbmthakur » Thu, Dec 30, 2021, 2:02 PM UTC

ransom1538 wrote: post #85

hu3 wrote: post #6

I expected more mentions of Vitess, which honestly looks like some kind of alien black magic from what I saw while consulting for a client this year. But I guess not much else happened to it other than PlanetScale.

All major companies are moving to Vitess. The battle is over. No one at scale uses Postgres.

Any reference for this?

Re: Databases in 2021: A Year in Review

#98

Post by hbarka » Thu, Dec 30, 2021, 3:02 PM UTC

I’m genuinely happy with Redshift for data warehousing purposes. For this I mean not-transactional data store. I don’t want to use the term OLTP or OLAP as it puts it in a purist’s camp. Sometimes I store 3NF normalized data and many times a flattened denormalized very large fact table and often times a model similar to star schema. I don’t have to worry about building indexes anymore, which was a real chore with row-store databases like Oracle, MySQL, SQL Server, or PostgreSql. MPP column-store databases have really been a game-changer for the enterprise. We’re talking billions of rows of data easily handled in the query plan.

Re: Databases in 2021: A Year in Review

#99

Post by _vvhw » Thu, Dec 30, 2021, 3:04 PM UTC

What are the distributed options for Postgres? What mechanisms are available to make it highly available i.e. with a distributed consensus protocol for strict serializability when failing over the primary? How do people typically deploy Postgres as a cluster?

1. Async replication tolerating data loss from slightly stale backup after a failover?

2. Sync replication tolerating downtime during manual failover?

3. Distributed consensus protocol for automated failover, high availability and no data loss, e.g. Viewstamped Replication, Paxos or Raft?

It seems like most managed service versions of databases such as Aurora, Timescale etc. are all doing option 3, but the open-source alternatives otherwise are still options 1 and 2?

Re: Databases in 2021: A Year in Review

#100

Post by apavlo » Thu, Dec 30, 2021, 3:06 PM UTC

newlisp wrote: post #30

Sytten wrote: post #11

Earlier quoted context omitted.

Couple of points: 1. Ottertune doesn't sell PostgreSQL services, they sell a database optimization service that happens to support PostgreSQL (and other databases like MySQL) 2. PostgreSQL is definitely gaining market shares and fast, see the db-engine graph [1], you can compare it to the oracle trend if you are not convinced [2] [1] https://db-engines.com/en/ranking_trend/system/PostgreSQL [2] https://db-engines.com…

they sell a database optimization service that happens to support PostgreSQL (and other databases like MySQL) A ML program that automatically tunes your production database in real-time. What could possibly go wrong?

We are very careful to make sure that we don't allow the tuning algorithms to make changes that could be detrimental to the correctness or availability of the database. This blog article describes some of the safeguards that we employ:

https://ottertune.com/blog/prevent-machine-learning-from-wre...

We also advise our customers to not point OtterTune at a production database right away.