So is https://github.com/obeli-sk/obelisk the Rust version of https://github.com/temporalio/temporal (Go)? Can you guys add a comparison between them on the site?
SQLite is all you need for durable workflows
341–350 of 413 posts
Re: SQLite is all you need for durable workflows
#342I've replaced all of these with Go + SQLite: 1. Intercom 2. Zendesk 3. Email marketing 4. Kanban 5. Todo 6. Our billing stack 7. Our issue tracker 8. Our forum 9. Uptime monitor 10. PagerDuty (clone) I have dozens of products I sell, so I thought: why not build everything ourselves? All of these run on the same server and use very little memory. I replaced all the SaaS tools we used with these. I also moved to dedica…
Thank you for sharing, if you don't mind can you share some Go+SQLite leanings as someone actually pulling this off?
1) How do you do backups? Do you use github.com/benbjohnson/litestream? CRON job backup with rsync?
2) Any issues with large databases and many clients? Is there a TPS or DB size where SQLite becomes problematic?
3) How do you deploy new binaries and safely shutdown the old instance? Caddy change to route to new binary + Go's HTTP server graceful-shutdown on old instance?
4) Do you use a pure-Go SQLite lib or one of the CGO libs?
Re: SQLite is all you need for durable workflows
#343Earlier quoted context omitted.
The reason SQLite is the most deployed is that it's used by Android.
…and iOS, and Windows, and Mac OS, and Boeing, and Sony, and Firefox and Chrome and Safari…
Yes, which goes in line with the argument that claiming that it's "the most deployed" as proof of superiority or suitability for any use case is equivalent to claiming the same for Internet Explorer. It's the most deployed because it's bundled in a lot of systems, not because people are purposefully using it as a DBMS.
Re: SQLite is all you need for durable workflows
#344Holy sticky header... It takes literally half the screen on mobile. Shit like this makes me wonder: do you even look at your website? At least sometimes?
Re: SQLite is all you need for durable workflows
#345Earlier quoted context omitted.
It sounds like you’re running this mostly on a single machine? Temporal gets much more complex with scale. Cassandra isn’t fun to manage. Ringpop and TChannel are hard to debug when things go wrong. The SQL backend support doesn’t support horizontally scaled replicas (just single instance) due to consistency requirements. Depending on how your code is written, modifying code baked into workflows becomes complex, as a…
In the last two years, we built (with a team of 15, now 100) a billion dollar business on top of Temporal that performs business critical applications for fortune 500 companies. We couldn't be happier with temporal. Determinism sucks, you do have to work hard and make everything idempotent in activities like we would for durable software anyway. The language we used was incorrect (Go) and has a lot of boilerplate com…
That is the real truth people are voicing when they say Temporal is heavy. They are really saying: Durable, reliable, distributed workloads are hard and it takes effort to manage! And that is true. I know of no systems that make that genuinely easy. It is a hard discipline. Maybe Temporal makes that harder than it should be, but I have no experience there.
There are no free lunches in this space. I have no idea how good or bad Temporal is since my usage is pretty small and isolated, but software rarely just works and impresses me and Temporal for my local machine orchestrating genuinely did. I think Netflix's conductor is another cool option, but I ended up with Temporal due to license.
Re: SQLite is all you need for durable workflows
#346Earlier quoted context omitted.
Is this bug logged?
Looks like https://github.com/benbjohnson/litestream/issues/1197 . Still open as of now, with a potential cause noted in the latest comment.
This is one of a bunch of issues that have been popping up since a vibe coder took over the bulk of development on this project. There's a (probably also AI generated) list of a big portion of the issues here: https://github.com/benbjohnson/litestream/issues/1221. That proposal has been open for a few months, and it seems (from my POV) unlikely to be resolved any time soon.
Re: SQLite is all you need for durable workflows
#347I don't understand this obsession with SQLite for real, production apps. SQLite is an embedded database, completely unsuitable for managing concurrency. This is what database _servers_ are for, e.g., Postgres, MySQL, etc. Their entire job is to allow you to modify data from multiple processes, on different machines, at the same time. This is a foundational principle of computer science. It seems to me that the "SQLit…
It's touted by the people who use the word "just" a lot. "Just use postgres" "Just use sqlite" "Juse use a monolith" "Just use sftp" "Just use an ec2 instance" Usually these people have flunked out of the school of (distributed system) hard knocks. They couldn't hack it and are retreating to familiar. The funny part is when one of those people fluke themselves into senior management when their saas takes off. Inevita…
That may exist but the opposite type of irrationality is much more common.
Scalability = success. We need to be "scalable" because that means we're successful right? Scalability = real engineering. I'm a real engineer so I need to design everything to be "scalable" because I'm so smart
>The funny part is when one of those people fluke themselves into senior management when their saas takes off.
>Inevitably they have to suck it up and hire experts in the same technologies that "no one needs".
Sounds like they were the wise ones to build something simple that achieved a high level of success.
Re: SQLite is all you need for durable workflows
#348Obelisk also supports Postgres. When you're using it with Postgres, what are the differences with DBOS? Are there any that would be significant
(I'm already using Postgres, so I don't really need a sqlite-based durable workflow engine, so looking to know how to choose between DBOS and Obelisk)
Re: SQLite is all you need for durable workflows
#349Earlier quoted context omitted.
At which point there’s little difference from any other database’s backup commands.
I don't think you can install "any other database" by pasting one file in a direcory somewhere? Even if you can produce such a backup with the same command.
Pretty much every embedded database since about 1988 has worked like that.
You say that being an embedded database isn't the main selling point, being contained within a single file is. But that's a completely normal feature of an embedded db, to the point that the one implies the other.
Re: SQLite is all you need for durable workflows
#350It's close enough that DBOS does support SQLite. [0] The default for prototyping is SQLite, but sure you can run it in production if you wanted. Obligatory list of workflow engines and libraries because it's such a common need that a lot have rolled their own. [1] [0] https://docs.dbos.dev/python/tutorials/database-connection [1] https://github.com/meirwah/awesome-workflow-engines
What would be the main differences between DBOS and Obelisk?