Reducing complexity by integrating through the database
1–10 of 45 posts
Re: Reducing complexity by integrating through the database
#2Stored procedures and the integration database have come back for our users in a big way. It would be great to hear examples of how others are applying this pattern with other databases and APIs.
Re: Reducing complexity by integrating through the database
#3[deleted]
Re: Reducing complexity by integrating through the database
#4[deleted]
Re: Reducing complexity by integrating through the database
#5Stored procedures and the integration database have come back for our users in a big way. It would be great to hear examples of how others are applying this pattern with other databases and APIs.
My favorite thing about this pattern is that it allows developers to build globally distributed applications while reasoning about them like a single, centralized monolith.
A lot of complicated practices that we’ve adopted to overcome the challenges of distributed systems just disappear when you have fast, strongly consistent transactional writes on a single data store. Either a thing happened or it didn’t. No need to wait around and see.
This matters even more as applications move from single failover to multi-region to edge computing with hundreds of locations of compute. How do you get consistent state out to each PoP?
You don’t, you integrate through the database.
Re: Reducing complexity by integrating through the database
#6The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is building using these “new ways” that are supposedly better.
Re: Reducing complexity by integrating through the database
#7The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is buildi…
I actually think this piece aligns with your thinking. Using a single, strongly consistent data store to eliminate the "train wreck of complexity" allows for a better developer experience while still delivering a responsive user experience. That matters at all kinds of scale, not just hyper.
Re: Reducing complexity by integrating through the database
#8My first two programming jobs out of college back in the early 2010's took the approach in this article. Albeit, with older technology.
This is bringing back old (bad) memories of the times where I was debugging stored procedures that called triggers that called the same stored procedures.
A lot of that was due to poor design and bad choices. Some of that was due to developers trying to fit processes and patterns into a language, i.e. SQL, that lacked the expressiveness for it.
I'm not a fan of this approach to say the least.
I'd be interested in hearing other's experiences with this though.
And I'm going to check out Fauna since it looks like a cool database and to see if anything has changed with this approach since I encountered it almost a decade ago.
Re: Reducing complexity by integrating through the database
#9My first two programming jobs out of college back in the early 2010's took the approach in this article. Albeit, with older technology. This is bringing back old (bad) memories of the times where I was debugging stored procedures that called triggers that called the same stored procedures. A lot of that was due to poor design and bad choices. Some of that was due to developers trying to fit processes and patterns int…
I remember those days too - my sins were T-SQL and I remember those headaches quite clearly.
One huge difference today is that you can actually unit-test user-defined functions and stored procedures using the same tooling you use for your application code. That lends itself naturally to integration testing, and so on.
I wouldn't necessarily recommend taking it to the property-based testing extreme like I did just to see whether it worked (it does).
Re: Reducing complexity by integrating through the database
#10While they quickly acknowledge that integrating by means of a database is widely regarded as an anti-pattern, the rest of the article doesn't really address why this wouldn't be an anti-pattern, other than to pretend that the major reasons for avoiding this pattern are deployment complexity and furnishing multi-region services.
> A customer pattern we see solves this problem, and it’s the integration database pattern. This pattern went out of style in the 90s along with Oracle servers in the closet and dedicated DBAs, but with technological advances like scalable multi-region transactions and serverless computing resources, its utility has returned.
Considering they are flogging a product, this feels especially dishonest to me.