Live data from Hacker News

TiDB – cloud-native, distributed SQL database written in Go

github.com

21–30 of 67 posts

Re: TiDB – cloud-native, distributed SQL database written in Go

#21

VC backed database company with a CLA on their "open source" project. Red flag. Is there a community fork yet?

Thank you for the warning. Whats the source on the VC part?

I don't share OP's hostility to VC-funded databases, but PingCAP is definitely VC-funded: https://www.pingcap.com/press-release/pingcap-the-company-be...

Re: TiDB – cloud-native, distributed SQL database written in Go

#22
post #3

TiDB is 8 years old now; what's new?

Yeah that and the actual core storage engine is a key-value store library called https://github.com/tikv/tikv which is written in Rust. I suspect the Go part only handles the replication component, MVCC, query planner, server etc.

Re: TiDB – cloud-native, distributed SQL database written in Go

#23
post #19
post #10

Earlier quoted context omitted.

I hear through the grapevine they’ve got a bunch of companies using it. I think Airbnb migrated to it.

There are quite a few. Pinterest, LinkedIn, Plaid, Mercari, and Rakuten shared their experiences with TiDB at PingCAP's annual event last year (2024) - https://www.pingcap.com/htap-summit/ . In the previous event (2023) there were speakers from Airbnb, Databricks, Flipkart, PayPay, and others sharing their experiences as well - https://www.pingcap.com/htap-summit/sept-2023/ Disclosure: Employee of PingCAP the company…

> PingCAP's annual event last year (2014)

Wasn't last year 2024?

Re: TiDB – cloud-native, distributed SQL database written in Go

#24
post #19
post #10

Earlier quoted context omitted.

I hear through the grapevine they’ve got a bunch of companies using it. I think Airbnb migrated to it.

There are quite a few. Pinterest, LinkedIn, Plaid, Mercari, and Rakuten shared their experiences with TiDB at PingCAP's annual event last year (2024) - https://www.pingcap.com/htap-summit/ . In the previous event (2023) there were speakers from Airbnb, Databricks, Flipkart, PayPay, and others sharing their experiences as well - https://www.pingcap.com/htap-summit/sept-2023/ Disclosure: Employee of PingCAP the company…

I think you mean 2024 and 2023...

Re: TiDB – cloud-native, distributed SQL database written in Go

#25
What is that first paragraph? "This implementation proves the CAP theorem as wrong"? Doesn't tell me what your drawbacks are. This is mongodb all over again. Will only ever consider if the customer really wants it. (My guess performance)

(Also, bold move to write a db without fully being able to manage memory).

Re: TiDB – cloud-native, distributed SQL database written in Go

#26

Hardly hear about TiDB. How is it faring and how does it stack up to more well know competition?

Same as before, excellent when you little little data and limited transactions. Once things get big and you lose sync, everything goes to shit.

Note that TiDB did subject itself to Jepsen testing (relatively) early. Here's their 2019 results:

https://jepsen.io/analyses/tidb-2.1.7

The devil is in the details, and anyone who is looking to implement TiDB for data correctness should read through not just this but other currently-open correctness-related Github issues:

e.g., https://github.com/pingcap/tidb/issues?q=is%3Aissue%20state%...

TiDB currently has 74 open issues related to "correctness," and has closed 163.

https://github.com/pingcap/tidb

Re: TiDB – cloud-native, distributed SQL database written in Go

#27

Earlier quoted context omitted.

how do you have doubts when nearly every significant company is running one, and distributed DBs are consistently proven to be the most correct (e.g. FDB, TigerBeetle)?

Sorry, should have clarified: I have doubts about their necessity (as I stated in another comment). Most tech companies have poor knowledge of proper data modeling and SQL, leading to poor schema design, and suboptimal queries. Combine that with the fact that networked storage (e.g. EBS) is the norm, and it’s no wonder that people think they need another solution. The amount of QPS you can get out of a single DB is s…

It all depends on what kind of queries you're running. I came from the OLTP market, where you're generally doing single-row operations. Basic CRUD. Single table work on denormalized data.

Now go to OLAP, and a single query might be doing multiple table joins. It might be scouring billions of records. It might need to do aggregations. Suddenly "millions of ops" might be reduced to 100 QPS. If you're lucky.

And yes, that's even using fast local NVMe. It's just a different kind of query, with a different kind of result set. YMMV.

Re: TiDB – cloud-native, distributed SQL database written in Go

#29
post #17

VC backed database company with a CLA on their "open source" project. Red flag. Is there a community fork yet?

How would you fund the open source development of a database? TiDB is Apache licensed, that should be enough no? Because if we ban all projects from VC-backed startups, you're going to clear your tech stack pretty quickly.

> How would you fund the open source development of a database?

I think the best way would be something similar to the Linux Foundation. Companies in need of a certain type of database would pool resources to develop and maintain it.

PostgreSQL was a community developed database that was funded in something approaching this way as most of the developers were either students or worked for someone who paid them while they worked on it.

> TiDB is Apache licensed, that should be enough no?

It's good enough to use it with the knowledge that it isn't truly free (libre) because the CLA gives the owning company the right to change that at any time. So you can use it but don't build a business around it or make it a critical part of your infrastructure as they might pull the rug out from under you.

Re: TiDB – cloud-native, distributed SQL database written in Go

#30
TiDB has four main components:

1. SQL front end nodes 2. Distributed shared nothing storage (TiKV) 3. Meta data server (PS) 4. TiFlash column store

1 and 3 are written in Go 2 is written in Rust and uses RocksDB 4 is written in C++

2 & 3 are graduated CNCF projects maintained by PingCAP.

Disclaimer: I work for PingCAP

Post reply on HN