Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reducing complexity by integrating through the database

fauna.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Reducing complexity by integrating through the database

#11

Post by chrisjc »

robsutter wrote: post #9
rc_hackernews wrote: post #8

My first two programming jobs out of college back in the early 2010's took the approach in this article. Albeit, with older technology. This is bringing back old (bad) memories of the times where I was debugging stored procedures that called triggers that called the same stored procedures. A lot of that was due to poor design and bad choices. Some of that was due to developers trying to fit processes and patterns int…

I remember those days too - my sins were T-SQL and I remember those headaches quite clearly. One huge difference today is that you can actually unit-test user-defined functions and stored procedures using the same tooling you use for your application code. That lends itself naturally to integration testing, and so on. I wouldn't necessarily recommend taking it to the property-based testing extreme like I did just to…

> you can actually unit-test user-defined functions and stored procedures using the same tooling you use for your application code

Can you give an example of how this is possible, or what tools/services it's possible with? I can understand how it would work on UDFs, but not entirely sure how it would be possible for stored procedures, esp when they're often times doing something more complex than simple read operations.

Re: Reducing complexity by integrating through the database

#13

Post by chrisjc »

orev wrote: post #6

The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is buildi…

While I see your point, it really felt like they were trying to empathize with "new" developers that might feel that way about doing things the "old way"... and that they may want to reconsider those feelings now that there are new technologies and services to overcome the limitations that existed in the "olden times".

Re: Reducing complexity by integrating through the database

#14

Post by chrisjc »

evanweaver wrote: post #2

Stored procedures and the integration database have come back for our users in a big way. It would be great to hear examples of how others are applying this pattern with other databases and APIs.

Since you're the one making the claim that they're making a come back, I'd love to hear your own personal story or some of the stories you've heard.

Personally I haven't noticed anything resembling a come back, but I'm certainly using them more than ever... and I'm loving it.

Re: Reducing complexity by integrating through the database

#15

Post by robsutter »

chrisjc wrote: post #11
robsutter wrote: post #9

Earlier quoted context omitted.

I remember those days too - my sins were T-SQL and I remember those headaches quite clearly. One huge difference today is that you can actually unit-test user-defined functions and stored procedures using the same tooling you use for your application code. That lends itself naturally to integration testing, and so on. I wouldn't necessarily recommend taking it to the property-based testing extreme like I did just to…

> you can actually unit-test user-defined functions and stored procedures using the same tooling you use for your application code Can you give an example of how this is possible, or what tools/services it's possible with? I can understand how it would work on UDFs, but not entirely sure how it would be possible for stored procedures, esp when they're often times doing something more complex than simple read operatio…

Sure - here's an example using Jest to test the logic in UDFs: https://github.com/fauna-labs/fql-utilities/blob/main/tests/...

You can use Jest's before[All] and after[All] to do any required setup and teardown per usual.

For Fauna, UDFs and stored procedures are synonymous.

Re: Reducing complexity by integrating through the database

#16

Post by rossdavidh »

Having worked on old systems that used this approach, I recall that it did have a lot of pluses. Now, for the minuses:

1) changing the language you use in a server (say, from PHP to Python) is big, but changing the language you use in your database (from SQL to anything else) is even more intimidating. If you are integrating through the database, this limitation matters more.

2) you need to have DBA's who are not only highly competent, but also have good people skills, since they will often be saying "no", or at least "not that way", and if they don't know how to do that in a constructive manner then it becomes a net productivity drain. Fortunately, where I worked that used this pattern, the DBA's had exceptional people skills as well as technical skills. This is, I am led to believe, not always the case.

Re: Reducing complexity by integrating through the database

#17

Post by iwintermute »

orev wrote: post #6

The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is buildi…

I know it's off-topic but maybe "MAANG level app" >> "MANGA" if we want go there?

Re: Reducing complexity by integrating through the database

#18

Post by cogman10 »

mdoms wrote: post #12

I think anyone with experience on this type of system will recoil in horror at the thought of building it intentionally today.

Tell me about it. This is the current system I maintain. While new products have (mostly) started using non-integrated DBs, all the legacy systems we have use a giant integrated db.

It SUCKS.

Let me count the ways.

- You can't update a table schema without updating and deploying a bunch of applications in sync (even if that's part of a stored proc). Which for something like this, means that getting out of this situation is WAY harder than getting into it.

- You end up putting WAY too much logic into the DB which makes it hard to ultimately figure out WHAT is supposed to happen

- DBs have TERRIBLE development stories. That's ultimately because the code and data all live in the same place and you can update code without any sort of revision control to help you understand or see a change that's been made to db schema (Forcing a bunch of painful process around updating DB capabilities).

- DBs are resource bottlenecks that SUCK to figure out how to scale out. Putting a bunch of apps into one DB complicates that process. Scaling a single app in a single DB is simply WAY easier.

- At least my db (but I assume a bunch of other DBs) have really crappy performance diagnostic tools. Further, the more complicated the queries against it, the more likely you are to go from "Hey, stats are making things fast" to "OMG, why is this thing taking 10 seconds to run now!". It's really bad when the only solution that seems to fix things is dumping stats.

I could MAYBE see something like this for a macro service dedicated to a domain, but I'd never build a complex system like this from scratch. Colocating apps in the same DB would have to be for some crazy performance reasons why bypassing a microservice makes sense. An exception, not the regular course of action (And I'd still hate it :) )

Re: Reducing complexity by integrating through the database

#19

Post by 6gvONxR4sf7o »

iwintermute wrote: post #17
orev wrote: post #6

The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is buildi…

I know it's off-topic but maybe "MAANG level app" >> "MANGA" if we want go there?

If you're going to rename F to M, why not rename G to A?

Re: Reducing complexity by integrating through the database

#20

Post by pred_ »

iwintermute wrote: post #17
orev wrote: post #6

The contempt towards the “old ways” in this piece is really irksome, especially since those “old ways” are perfectly acceptable for 95% of applications (you only need this hyper scale stuff if you’re running a MAANG level app). The rest of us still doing things the “old way” are perfectly happy to have simple systems that run reliably, as we sit and watch the complete train wreck of complexity everyone else is buildi…

I know it's off-topic but maybe "MAANG level app" >> "MANGA" if we want go there?

But if F->M, then why not G->A? MAAAN.
Post reply on HN