Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TigerBeetle is a most interesting database

amplifypartners.com

Post reply on HN
Print view View original

151–160 of 228 posts

Re: TigerBeetle is a most interesting database

#151

Post by parthdesai »

sgarland wrote: post #80

While I am a fan of TigerBeetle’s general stance on correctness, coding practices, and their desire to hyper-specialize, I have some critiques on the post. I think the paragraph about multi-node is a bit misleading. Contrary to what cloud native folk will tell you, a single beefy DB, well-tuned and with a connection pooler, can serve a dizzying amount of QPS just fine. At a former employer, during a maintenance perio…

> My other mild criticism is in the discussion on TigerBeetle’s consensus: yes, it seems quite clever and has no other dependencies, but it’s also not trying to deal with large rows. When you can fit 8,190 transactions into a 1 MiB packet that takes a single trip to be delivered, you can probably manage what would be impossible for a traditional RDBMS.

Isn't that the point? They're saying to separate out transactions workload from other workloads. They're not saying they'll replace your OLGP db, you remove transactionally important data into another db.

It's something similar that we see with another db: https://turbopuffer.com/

Re: TigerBeetle is a most interesting database

#152

Post by jandrewrogers »

Normal_gaussian wrote: post #34
dakiol wrote: post #27

Earlier quoted context omitted.

I disagree. If we are talking about distributed systems in which we have N different databases, then distributed transactions are left as an exercise to the reader (that’s why we have things like Sagas). Within a single machine, yeah, relational dbs still work like a charm.

The other interesting thing to consider is that machines are a lot more powerful in all dimensions than they were 30 years ago - so many tasks that would have required distributed systems no longer do.

The caveat is that the hardware constraints have fundamentally changed, so database architectures from 30 years ago are effectively incapable of properly using modern hardware environments.

Databases have not been bottlenecked on storage bandwidth in a long time but most databases are designed as if this was still the case. Optimizing for memory bandwidth, the current bottleneck, leads to substantially different architectures than are commonly deployed.

Re: TigerBeetle is a most interesting database

#153

Post by mbrock »

WhitneyLand wrote: post #84
cma256 wrote: post #54

Earlier quoted context omitted.

Its a financial database built for use cases where this invariant holds and built for enabling new use cases where this invariant prevented businesses from expanding into new industries. The creator says as much: > Without much sweat for general purpose workloads.

You’re changing the subject to the company’s mission when the concern was about a specific claim made.

A specific claim about OLTP processing under contention. Or how would you characterize the specific claim, specifically?

Re: TigerBeetle is a most interesting database

#155

Post by xxpor »

dangoodmanUT wrote: post #85

I really like the work Joran and his team have done around DST, distributed systems awareness, and performance practices. I especially love the craziness around no dependencies (well, could you consider an underlying OS a dependecy?). But I've always felt the way they treat normal OLTP (they call OLGP) seems unfair. For example, comparisons using clear sub-optimal interactive SQL transactions for financial workloads,…

>well, could you consider an underlying OS a dependecy?

I've worked on systems that ran entirely in memory, and continued running during kexec. You can't do syscalls. So yeah :)

Re: TigerBeetle is a most interesting database

#156

Post by threatofrain »

wiradikusuma wrote: post #99
vjerancrnjak wrote: post #35

Earlier quoted context omitted.

But Cloudflare Workers or AWS Lambda setup would not work anyway with any db? * spawning 1000 workers all opening a connection to a db, * solved by service/proxy in front of db, * proxy knows how to reach db anyway, let's do private network and not care about auth

Cloudflare has its own DB (D1, Sqlite-derived), but you can also connect with PostgreSQL using their adapter (Hyperdrive). I have used both, they're okay.

I wouldn't recommend D1 for now due to its harsh storage limitations (10 GB).

Re: TigerBeetle is a most interesting database

#157

Post by sgt »

samlambert wrote: post #147
sgt wrote: post #14

Earlier quoted context omitted.

I think the point is that sharding won't really help that much since transactions will happen across all or most shards, and then you have certain accounts that will be more active than others.

if your sharding schema is designed properly you will avoid cross shard transactions.

How so? Any account may transact with any other account - regardless which shard it resides in.

Re: TigerBeetle is a most interesting database

#158

Post by jorangreef »

caust1c wrote: post #77
wiradikusuma wrote: post #20

We were considering TigerBeetle, but found blockers: * We use Cloudflare Workers. TigerBeetle client app is not supported. It might work using Cloudflare Containers, but then the reason we use Cloudflare is for the Workers. --> https://github.com/tigerbeetle/tigerbeetle/issues/3177 * TigerBeetle doesn't support any auth. It means the containing server (e.g. a VPS) must restrict by IP. Problem is, serverless doesn't h…

Wait, what? A database in 2025 doesn't support any kind of auth? A financial database? WTF? C'mon folks, the least you can do is put a guide for adding an auth proxy or auth layer on your site. Particularly since you don't use HTTP (cant easily tell from the docs, I'm assuming), then folks are going to be left wondering: "well how the hell do I add an auth proxy without HTTP" and just put it on the open internet...

Joran from TigerBeetle here!

TigerBeetle is our open source contribution. We want to make a technical contribution to the world. And we have priorities on the list of things we want to support, and support properly, with high quality, in time.

At the same time, it's important I think that we encourage ourselves and each other here, you and I, to show respect to projects that care about craftsmanship and doing things properly, so that we don't become entitled and take open source projects and maintainers in general for granted.

Let's keep it positive!

Re: TigerBeetle is a most interesting database

#159

Post by jorangreef »

dangoodmanUT wrote: post #85

I really like the work Joran and his team have done around DST, distributed systems awareness, and performance practices. I especially love the craziness around no dependencies (well, could you consider an underlying OS a dependecy?). But I've always felt the way they treat normal OLTP (they call OLGP) seems unfair. For example, comparisons using clear sub-optimal interactive SQL transactions for financial workloads,…

Hey Dan, appreciate the compliment.

Do you think it would be more fair to suggest that OLTP workloads have 0% contention? That debit/credit sets never intersect?

In practice, we're seeing that our customers are already some of the largest brokerages, wealth managements or exchanges, in their jurisdictions, even national projects, and all of them with decades of Postgres etc. experience, some even running installations with 200 Postgres/MySQL machines backing their sharded core ledger.

They're not junior engineers. For example, they know about stored procedures, but the problem of concurrency runs deeper, into the storage engine itself.

At least for our customers, for the seasoned payments engineers, OLTP contention is a killer. They're tired of scaling outside the DBMS, and the complexity of expensive reconciliation systems. Several of them are even leaving their Chief Architect positions and starting TB-related startups—I know of at least 4 personally, from fintech brands you will recognize, and probably use.

I hope the spirit of our talks, trying to get the big ideas across, to make people aware of contention and Amdahl's Law, is clear, and that you take it in good faith, to the extent that we show it ourselves.

Re: TigerBeetle is a most interesting database

#160

Post by noir_lord »

blenderob wrote: post #134

As nice as TigerBeetle is, do keep in mind that this article is written by an investment company that has invested in TigerBeetle: https://www.amplifypartners.com/blog-posts/our-investment-in...

Yeah I'm not going to say what I think of the way it's written but suffice to say that was obvious from how it was written.
Post reply on HN