31–37 of 37 posts
Like it uses strings (OLAP works way better over integral data, it sucks at strings) or that it's easy to scale.
It is easy-ish under fixed queries (classic MOLAP for example) but not over arbitrary queries and frequent updates, then it degenerates to a problem much worse than OLTP.
Earlier quoted context omitted.
If you think that my opinion is not worth listening to, i.e. that I am wrong, would you mind elaborating why? There is a real opportunity to sway my opinion here, because I am not unsure. I could just be crazy. I don't know anymore. But, generally, I don't think that you necessarily have to have multiple back-end instances, and that if you have multiple back-end instances, that you will necessarily have race conditio…
Well, your opinion doesn't consider many real world factors. (It's also worth noting that you toned down your response to me with hedging language... "generally", "necessarily") No matter how good a particular server it, it isn't immune to power outages, fiber cuts, fires, basic hardware failure or even just downtime for basic updates (os, application deployment, etc). As soon as there is any sort of cost with downti…
My point is that, by default ("generally"), there is no need to run multiple instances. But I admit that sometimes it can be justified, such as if you (truly) need zero-downtime deployment or N+1 redundancy (which is questionable, because, as senko pointed out, services usually go down for reasons that aren't a power outage or a fire). My complaint wasn't really about this anyway, sorry for not being clear. I mentioned that my company runs multiple services "for absolutely no fucking reason," but I did not state their intention, which is not zero-downtime deployment or redundancy, but rather "scalability," and I find this pointless because a single $5 VPS could easily handle many times the amount of traffic their IO-bound app receives. The end result is that they now must always think about the implications of running multiple instances, and failure to do that properly creates race conditions and other kinds of obscure bugs.
On the other hand, if you only run a single instance, reasoning about the system becomes much easier, and even though that does not eliminate the possibility of a race condition, it becomes much harder to create one. I also like the architectural benefits of a single instance system, such as being able to keep transient state in memory worry-free.
Earlier quoted context omitted.
To a first approximation, yes. But, why ? And for up to how many hundred terabytes of data can you get away with the single beefy server? Provided you make what design choices? Which leads to the real takeaway which is "Tiger Style": https://tigerstyle.dev/ which I am partial to, along with Rich Hickey's "Hammock Driven Development" https://www.youtube.com/watch?v=f84n5oFoZBc "Tiger on Hammock" will absolutely smoke…
> But, why? To keep things simple. My current company is running multiple instances of back-end services for absolutely no fucking reason, and I had to fix numerous race condition bugs for them. I had an interview with a startup where, after I asked why they were using distributed DynamoDB locks in a monolith app with only a single instance running, the person said "it works for us" and got defensive. Later they told…
Oh I am so with this. The "why" I posed was in context of TigerBeetle's design choices to solve for high-contention OLAP.
Sorry, I framed my question loosely --- too much implicit context.
Me personally, I'm learning from the lesson of TigerBeetle and others, and just using SQLite for my multi-ten-gigabyte ambitions :D
This looks like yet another basic key value store. Benchmarking is a complicated problem, but FoundationDB claims 300,000 reads per second on a single core. TigerBeetle claims 100k-500k TPS on... Some kind of hardware? https://apple.github.io/foundationdb/benchmarking.html
Both are great
I have tried to use tiger beetle in production. haven't been successful yet. nice stuff, multi master replication. user API, super small. doubts about how to do streaming backup. after studying the API and doing some spike architectures I come to the conclusion (I may be wrong): tiger beetle is awesome to keep the account balance. that's it. because you pretty much get the transactions affecting and account and IIRC…
> how much money out user accounts have in this microsecond My understanding is that if you want aggregations or sub accounts then you need to duplicate transactions and maintain them yourself. This may seem like it would be annoying, but I suspect it would mostly be a matter of code organization. I expect typical TigerBeetle (TB) clients will develop and maintain an application-level library that builds up a set of…
Great video! Love the concepts discussed, and the tigerstyle mentioned by adityaathealye. This is such a hard problem. For Halo 4, I architected a system for our telemetry that that had to support subsecond roundtrip user lifetime aggregation. With up to 400ms one-way latencies, that left us with 200ms client + server time to manage lifetime counts. 60ms on the client for batching and network stack delays, and 80ms i…