Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TiDB – cloud-native, distributed SQL database written in Go

github.com

Post reply on HN
Print view View original

31–40 of 67 posts

Re: TiDB – cloud-native, distributed SQL database written in Go

#31

Post by travem »

normie3000 wrote: post #23
travem wrote: post #19

Earlier quoted context omitted.

There are quite a few. Pinterest, LinkedIn, Plaid, Mercari, and Rakuten shared their experiences with TiDB at PingCAP's annual event last year (2024) - https://www.pingcap.com/htap-summit/ . In the previous event (2023) there were speakers from Airbnb, Databricks, Flipkart, PayPay, and others sharing their experiences as well - https://www.pingcap.com/htap-summit/sept-2023/ Disclosure: Employee of PingCAP the company…

> PingCAP's annual event last year (2014) Wasn't last year 2024?

Yes, I am still getting up to speed after a long break for Christmas and New Year!

Re: TiDB – cloud-native, distributed SQL database written in Go

#32

Post by std_reply »

xmichael909 wrote: post #6
willvarfar wrote: post #4

Hardly hear about TiDB. How is it faring and how does it stack up to more well know competition?

Same as before, excellent when you little little data and limited transactions. Once things get big and you lose sync, everything goes to shit.

This is quite an ignorant comment. TiDB routinely handles 100s of TB of data. Go watch LinkedIn’s presentation on why they choose TiDB as their strategic db going forward.

Re: TiDB – cloud-native, distributed SQL database written in Go

#34

Post by travem »

lambda wrote: post #24
travem wrote: post #19

Earlier quoted context omitted.

There are quite a few. Pinterest, LinkedIn, Plaid, Mercari, and Rakuten shared their experiences with TiDB at PingCAP's annual event last year (2024) - https://www.pingcap.com/htap-summit/ . In the previous event (2023) there were speakers from Airbnb, Databricks, Flipkart, PayPay, and others sharing their experiences as well - https://www.pingcap.com/htap-summit/sept-2023/ Disclosure: Employee of PingCAP the company…

I think you mean 2024 and 2023...

Yes, corrected!

Re: TiDB – cloud-native, distributed SQL database written in Go

#35

Post by sgarland »

PeterCorless wrote: post #27
sgarland wrote: post #12

Earlier quoted context omitted.

Sorry, should have clarified: I have doubts about their necessity (as I stated in another comment). Most tech companies have poor knowledge of proper data modeling and SQL, leading to poor schema design, and suboptimal queries. Combine that with the fact that networked storage (e.g. EBS) is the norm, and it’s no wonder that people think they need another solution. The amount of QPS you can get out of a single DB is s…

It all depends on what kind of queries you're running. I came from the OLTP market, where you're generally doing single-row operations. Basic CRUD. Single table work on denormalized data. Now go to OLAP, and a single query might be doing multiple table joins. It might be scouring billions of records. It might need to do aggregations. Suddenly "millions of ops" might be reduced to 100 QPS. If you're lucky. And yes, th…

Not sure why you think OLTP doesn’t also do complex joins. In a properly normalized schema, you’ll likely have many.

But yes, OLAP is of course its own beast, and most DBs are suited for one or the other.

Re: TiDB – cloud-native, distributed SQL database written in Go

#36

Post by travem »

nuphonenuacc wrote: post #25

What is that first paragraph? "This implementation proves the CAP theorem as wrong"? Doesn't tell me what your drawbacks are. This is mongodb all over again. Will only ever consider if the customer really wants it. (My guess performance) (Also, bold move to write a db without fully being able to manage memory).

I do want to clarify a few points, on the project page it does provide the following information:

> Distributed Transactions: TiDB uses a two-phase commit protocol to ensure ACID compliance, providing strong consistency. Transactions span multiple nodes, and TiDB's distributed nature ensures data correctness even in the presence of network partitions or node failures.

> High Availability: Built-in Raft consensus protocol ensures reliability and automated failover. Data is stored in multiple replicas, and transactions are committed only after writing to the majority of replicas, guaranteeing strong consistency and availability, even if some replicas fail. Geographic placement of replicas can be configured for different disaster tolerance levels.

See https://github.com/pingcap/tidb?tab=readme-ov-file#key-featu...

Correctness has been a focus for a long time for TiDB, including working on passing Jepsen Tests back in 2019, see https://www.pingcap.com/blog/tidb-passes-jepsen-test-for-sna... and https://jepsen.io/analyses/tidb-2.1.7

Disclosure: Employee of PingCAP the company behind TiDB

Re: TiDB – cloud-native, distributed SQL database written in Go

#37

Post by fipar »

nuphonenuacc wrote: post #25

What is that first paragraph? "This implementation proves the CAP theorem as wrong"? Doesn't tell me what your drawbacks are. This is mongodb all over again. Will only ever consider if the customer really wants it. (My guess performance) (Also, bold move to write a db without fully being able to manage memory).

I can't find that first paragraph, and it seems that readme file was last modified 3 months ago, though you're using quotes, so perhaps you are paraphrasing?

The first paragraph I find under Key Features is:

Distributed Transactions: TiDB uses a two-phase commit protocol to ensure ACID compliance, providing strong consistency. Transactions span multiple nodes, and TiDB's distributed nature ensures data correctness even in the presence of network partitions or node failures.

It claims to ensure correctness in the presence of partitions, not availability, so I don't think that's a claim that CAP is wrong? I would expect it to be unavailable if it can't provide a consistent response.

I don't know about memory management. It seems go may not be used for the storage layer, but in any case, it wouldn't be the first attempt at a DB on a garbage collected runtime. I can think of at least Cassandra on the jvm and CockroachDB also on Go. I do prefer that DBs do their own memory management (as well as storage) but this is not as unique as it used to be though.

Re: TiDB – cloud-native, distributed SQL database written in Go

#38

Post by std_reply »

sgarland wrote: post #9
xmichael909 wrote: post #6

Earlier quoted context omitted.

Same as before, excellent when you little little data and limited transactions. Once things get big and you lose sync, everything goes to shit.

They claimed they successfully onboarded a customer with 1 PB of data at KubeCon NA. I have strong doubts about distributed DBs in general, but I also can’t see them blatantly lying in a talk.

TiDB is running core banking services too.

I think people’s idea of scale and operating at scale is limited to their experience.

You can get MySQL to run at any scale, look at Meta and Shopify. Operational complexity at that scale is a different story.

Distributed databases reduce a lot of the operational complexity. To take one example:

Try a DDL on a 5 TB table in any replicated MySQL topology of choice and compare it with TiDB’s Distributed execution framework.

Re: TiDB – cloud-native, distributed SQL database written in Go

#39

Post by std_reply »

eikenberry wrote: post #29
hbbio wrote: post #17

Earlier quoted context omitted.

How would you fund the open source development of a database? TiDB is Apache licensed, that should be enough no? Because if we ban all projects from VC-backed startups, you're going to clear your tech stack pretty quickly.

> How would you fund the open source development of a database? I think the best way would be something similar to the Linux Foundation. Companies in need of a certain type of database would pool resources to develop and maintain it. PostgreSQL was a community developed database that was funded in something approaching this way as most of the developers were either students or worked for someone who paid them while t…

TiKV the Core Storage scalable component is a CNCF graduated projected. PingCAP cannot change the license even if it wanted to.

Re: TiDB – cloud-native, distributed SQL database written in Go

#40

Post by eikenberry »

russell_h wrote: post #21
zhouzhao wrote: post #16

Earlier quoted context omitted.

Thank you for the warning. Whats the source on the VC part?

I don't share OP's hostility to VC-funded databases, but PingCAP is definitely VC-funded: https://www.pingcap.com/press-release/pingcap-the-company-be...

The VC part isn't 100% negative but it generally comes with the strings attached that highly incentivizes fast growth. Fast growth kills any company culture you have and converts you into yet another company that views all employees as cost-centers. IE. it destroys companies as good places to work. If you could get VCs to fund companies to grow slowly into stable, small to mid-sized companies I would have no problem with them.

A side aspect of this is that they also destroy any motivation for developing free software other than as a marketing tool, a way to drive growth. While that is perfectly fine for companies to do this, it has long term consequences that lead to bad behavior that mean you should be extra wary of relying on them.

Post reply on HN