Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TigerBeetle is a most interesting database

amplifypartners.com

Post reply on HN
Print view View original

101–110 of 228 posts

Re: TigerBeetle is a most interesting database

#101

Post by cmrdporcupine »

justinclift wrote: post #13

> In less than a decade, the world has become at least three orders of magnitude more transactional. And yet the SQL databases we still use to power this are 20-30 years old. Can they hold up? Errr yes. Without much sweat really. Just because something started ~30 years ago doesn't mean it hasn't updated with the times, and doesn't mean it was built on bad foundations.

Yeah, SQL is not the issue. At least not mostly.

The relational model has shown itself to be exactly the flexible and powerful model that Codd said it was, even in its relatively-debased from in SQL.

In fact the potential of it as a universal and flexible data model that abstracts away storage still remains to be fully unlocked.

Now as for existing SQL databases, yes, many of them were built on foundational assumptions about the nature of memory and secondary storage that no longer hold true.

Many of us in this industry still have our heads stuck in the past world of spinny spinny magnetic disks on platters with heads and cylinders and grinding noises. Real world hardware has moved on. We have a couple orders of magnitude higher IOPS generally available than we did just 10-15 years ago.

So I'm excited about products like CedarDB and others (Umbra before it, etc) that are being built on a foundation from day one of managing with hybrid in-memory/disk.

Throwing out SQL is not really the recipe for "performance" and the lessons of separating the storage system from the data model is still key, since the 1960s.

I am willing to grant that a specialized transaction/ledger system like TigerBeetle might have its place as an optimization strategy in a specific industry/domain, but we should not make the mistake of characterizing this as a general problem with the relational model and data storage broadly.

Re: TigerBeetle is a most interesting database

#102

Post by wiradikusuma »

caust1c wrote: post #77
wiradikusuma wrote: post #20

We were considering TigerBeetle, but found blockers: * We use Cloudflare Workers. TigerBeetle client app is not supported. It might work using Cloudflare Containers, but then the reason we use Cloudflare is for the Workers. --> https://github.com/tigerbeetle/tigerbeetle/issues/3177 * TigerBeetle doesn't support any auth. It means the containing server (e.g. a VPS) must restrict by IP. Problem is, serverless doesn't h…

Wait, what? A database in 2025 doesn't support any kind of auth? A financial database? WTF? C'mon folks, the least you can do is put a guide for adding an auth proxy or auth layer on your site. Particularly since you don't use HTTP (cant easily tell from the docs, I'm assuming), then folks are going to be left wondering: "well how the hell do I add an auth proxy without HTTP" and just put it on the open internet...

Exactly my thoughts, and yes, it's not HTTP. Not a big deal, I guess, if they explain how to work around it...

Re: TigerBeetle is a most interesting database

#103

Post by jorangreef »

big_hacker wrote: post #43
dakiol wrote: post #27

Earlier quoted context omitted.

I disagree. If we are talking about distributed systems in which we have N different databases, then distributed transactions are left as an exercise to the reader (that’s why we have things like Sagas). Within a single machine, yeah, relational dbs still work like a charm.

If it's gonna fit on one machine I'm picking Postgres, Mongo or Redis before TigerBeetle and let's be honest it's not the difference between the 4 that will make it not fit on one machine.

And to be fair, those are all great general purpose solutions. For example, you could also use Postgres or Redis as queue, instead of say Redpanda or Kafka.

But our customers need separation of concerns in their architecture. While they could put the cash in the general purpose filing cabinet, they actually want the separation of concerns between OLGP system of reference (string database, i.e. PG) in the control plane, and OLTP system of record (integer/counting database, i.e. TB) in the data plane.

Re: TigerBeetle is a most interesting database

#104

Post by oefrha »

tux3 wrote: post #44
jorangreef wrote: post #33

Earlier quoted context omitted.

Joran from TigerBeetle! Without much sweat for general purpose workloads. But transaction processing tends to have power law contention that kills SQL row locks (cf. Amdahl’s Law). We put a contention calculator on our homepage to show the theoretical best case limits and they’re lower than one might think: https://tigerbeetle.com/#general-purpose-databases-have-an-o...

>Traditional SQL databases hold locks across the network; under Amdahl's Law, even modest contention caps write throughput at ≈100–1,000 TPS In fact large real world systems are not limited to 100-1000 TPS, or even 10 kTPS as the calculator tries to suggest. That's not because Amdahl's law is wrong, the numbers you're plugging in are just wildly off, so the conclusions are equally nonsensical. There might be some spe…

They’re talking about 100-1000 TPS for transactions all locking a single row, which is not wrong, just not reflective of most workloads. They’re not talking about TPS of the entire database with simultaneous operations on many independent rows. This should be reasonably clear in context, but of course when you publish grandiose claims (when viewed in isolation) and very vague graphs backing said claims people won’t be happy.

TFA contextualizes this better:

> This gets even worse when you consider the problem of hot rows, where many transactions often need to touch the same set of “house accounts”.

Re: TigerBeetle is a most interesting database

#105

Post by surajrmal »

ksec wrote: post #83

>Most teams write code fast. TigerBeetle tries to write code slow. Most teams treat testing as a necessary evil. TigerBeetle is built entirely on Deterministic Simulation Testing (DST). Most teams build their software on top of loads of other software. TigerBeetle has zero dependencies. I believe that was more like the norm 25+ years ago. Before Google and Facebook brought in the move fast and break things mentality…

Move fast and break things was the Facebook motto. I've never really felt it was a Google embraced concept. What Google's values have always lied with investing heavily on testing to ensure folks can be productive by knowing that if they would break something a test would catch it.

That said it's good to make sure you're building for requirements that exist. Engineers have a habit of inventing requirements and causing delays unnecessarily. Building something and placing it in the hands of users so they can give you feedback which you can react to is far more valuable than building products in a bubble.

Re: TigerBeetle is a most interesting database

#106

Post by kevinqi »

TZubiri wrote: post #78
flir wrote: post #23

> They keep assertions enabled in production. Never understood why we turn those off. An assert failing in prod is an assert that I desperately want to know about. (That "never understood" was rhetorical).

Because the very raison d'etre of asserts is to be a compile time/test time check. If you wanted them to run in production, you wouldn't use an assert, you would just run an if. Otherwise what's an assert to you? A mildly convenient sugar syntax alternative to if statements?

right, it is just syntactic sugar, but if that wasn't helpful then why have it in dev either? I find it more confusing to have asserts be stripped, which creates an implicit dev/prod discrepancy

Re: TigerBeetle is a most interesting database

#107

Post by jorangreef »

wiradikusuma wrote: post #20

We were considering TigerBeetle, but found blockers: * We use Cloudflare Workers. TigerBeetle client app is not supported. It might work using Cloudflare Containers, but then the reason we use Cloudflare is for the Workers. --> https://github.com/tigerbeetle/tigerbeetle/issues/3177 * TigerBeetle doesn't support any auth. It means the containing server (e.g. a VPS) must restrict by IP. Problem is, serverless doesn't h…

You should chat to our solutions team, who'd be able to help you solve auth etc. at the logical level with end-to-end encryption, something we typically do for our customers.

Re: TigerBeetle is a most interesting database

#109

Post by jorangreef »

CGMthrowaway wrote: post #31
SchwKatze wrote: post #10

Pretty much agree, I honestly think that TB team should invest some more effort on evangelizing the double-entry model for non-financial scenarios, like managing stocks, booking a show ticket etc. The 1000x API improvement has been done, now people must know how to use it

It sounds awesome. I'm an a analytics guy using a lot of SQL - not a coder. Though I understand the OP writeup and the purported performance advantages. Can someone explain: a) what Tigerbeetle data looks like in practice? Assuming it doesn't look like a regular table b) how you use it, if you can't write sql queries? c) separately, curious what double-entry would look like for stocks, tickets etc. E.g. I'm a venue a…

As one example, Rene Schallner made a pretty cool ticketing demo called TigerFans: https://tigerfans.io

Re: TigerBeetle is a most interesting database

#110

Post by jorangreef »

moffers wrote: post #86
jorangreef wrote: post #18

Earlier quoted context omitted.

Depending on contention and RTT, as a specialized OLTP DBMS, TB can do roughly 1000-2000x more performance than a single node OLGP DBMS (cf. the live demo in the talk above)… but also with strict serializability. You don’t need to sacrifice correctness or real-time resolution, and that’s important. For example, if you need to do real-time balance checks.

TB seems really awesome, but is there non-DebitCredit use cases where it can be applied effectively? I like trying to find off-label uses for cool technology

Thanks! Check out https://tigerfans.io
Post reply on HN