Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Future Database

principles.planetscale.com

Post reply on HN
Print view View original

31–40 of 40 posts

Re: The Future Database

#31

Post by gnulinux »

Scarbutt wrote: post #17
rmbyrro wrote: post #2

My dream was having a database like AWS DynamoDB, but with analytical capabilities built-in the API.

Doing application programming in Dynamodb is a nightmare.

I personally found DynamoDB enjoyable but the trick is you need to understand DynamoDB gives you exactly no way to have analytics, so you have to have a separate and robust story for analytics e.g. we periodically dump our DynamoDB into a RedshiftDB so we get best of the both worlds. I personally find it enjoyable to program, and had a better experience than when everything was Postgres.

Re: The Future Database

#32

Post by ertian »

throwusawayus wrote: post #30
ertian wrote: post #29

Earlier quoted context omitted.

I suspect the intention is something more like IPFS ( https://ipfs.io ), built on some distributed data structure like a DHT. With that in mind: - "Infinitely scalable" in the sense of the internet, I suppose? If we had a way of paying for storage and indexing in a decentralized way, then you could switch from one 'provider' to another, or host your own data. Data would not be siloed in the way it is by AWS. - I assu…

> I suspect the intention is something more like IPFS ( https://ipfs.io ), built on some distributed data structure like a DHT. the company's products are all built on top of sharded mysql .. unless they plan to throw away everything they have done so far, i do not think these assumptions are correct!

Ah, fair enough. I didn't look into the company, I've just come across suggestions along this line.

Re: The Future Database

#33

Post by rmbyrro »

gnulinux wrote: post #31
Scarbutt wrote: post #17

Earlier quoted context omitted.

Doing application programming in Dynamodb is a nightmare.

I personally found DynamoDB enjoyable but the trick is you need to understand DynamoDB gives you exactly no way to have analytics, so you have to have a separate and robust story for analytics e.g. we periodically dump our DynamoDB into a RedshiftDB so we get best of the both worlds. I personally find it enjoyable to program, and had a better experience than when everything was Postgres.

Same experience.

From an OLTP standpoint, the limitations to query the data are actually amazing, because it forces you to think about the right questions from the beginning.

On more flexible DBMS like Postgres, it's very very easy to shoot yourself in the foot.

I find it easier to understand and take into consideration the DynamoBD constraints (explicit) than SQL shortcomings (implicit).

Re: The Future Database

#34

Post by Scarbutt »

rmbyrro wrote: post #33
gnulinux wrote: post #31

Earlier quoted context omitted.

I personally found DynamoDB enjoyable but the trick is you need to understand DynamoDB gives you exactly no way to have analytics, so you have to have a separate and robust story for analytics e.g. we periodically dump our DynamoDB into a RedshiftDB so we get best of the both worlds. I personally find it enjoyable to program, and had a better experience than when everything was Postgres.

Same experience. From an OLTP standpoint, the limitations to query the data are actually amazing, because it forces you to think about the right questions from the beginning. On more flexible DBMS like Postgres, it's very very easy to shoot yourself in the foot. I find it easier to understand and take into consideration the DynamoBD constraints (explicit) than SQL shortcomings (implicit).

because it forces you to think about the right questions from the beginning.

For many apps thinking that you can have the right questions from the beginning is delusional.

Re: The Future Database

#36

Post by gnulinux »

Scarbutt wrote: post #34
rmbyrro wrote: post #33

Earlier quoted context omitted.

Same experience. From an OLTP standpoint, the limitations to query the data are actually amazing, because it forces you to think about the right questions from the beginning. On more flexible DBMS like Postgres, it's very very easy to shoot yourself in the foot. I find it easier to understand and take into consideration the DynamoBD constraints (explicit) than SQL shortcomings (implicit).

because it forces you to think about the right questions from the beginning. For many apps thinking that you can have the right questions from the beginning is delusional.

> For many apps thinking that you can have the right questions from the beginning is delusional.

Strong disagree, I think you might be approaching this from a traditional angle where analytics questions are also questions of the system. In DynamoDB you only focus on access patterns you need for system to work real-time, without bothering with analytics (as stated above). Any question you want to ask on data outside of the operational code is out of the scope.

Re: The Future Database

#38

Post by dravita »

thdxr wrote: post #35
rmbyrro wrote: post #2

My dream was having a database like AWS DynamoDB, but with analytical capabilities built-in the API.

been pairing DynamoDB with rockset to solve this

Try SingleStoreDB for the combined DynamoDB workloads and analytical workloads in a single database tech, see https://usefathom.com/blog/ditched-dynamodb

Re: The Future Database

#40

Post by dravita »

azurezyq wrote: post #8
rmbyrro wrote: post #2

My dream was having a database like AWS DynamoDB, but with analytical capabilities built-in the API.

I think you are talking about HTAP( https://en.wikipedia.org/wiki/Hybrid_transactional/analytica... ). There are already decent products on the market. But don't expect 100% analytical capabilities like BigQuery / Snowflake. There are tradeoffs. If your workload fits the model well (mostly OLTP, but occasionally some with heavier aggregations, etc.), it would be awesome. Example: TiDB https://docs.pingcap.com/tidb/de…

When it comes to HTAP workloads, SingleStoreDB tackled that years ago and has been the leader in that category of workloads. In recent years, the product has expanded to handle broader workloads and use cases. https://www.singlestore.com/resources/research-paper-cloud-n...
Post reply on HN