Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does anyone integrate with their customers' DB directly?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 48 posts

Does anyone integrate with their customers' DB directly?

#1

Post by johnyeocx »

Hey all, integrations with third-party SaaS vendors are super common these days, especially with all the AI tools complementing and enriching SaaS platforms, but as the title states, I'm curious to hear if anyone integrates directly with their customers' data store like Supabase, Snowflake, etc.

In my last startup, we were building a chargeback management tool for fintechs, and essentially whenever a chargeback came in, we'd only get the transaction ID, so we'd have to query the fintech's database to search for that transaction ID and pull additional data points like name of seller, email of seller, etc.

Now, when we did this, it was more of a scrappy workaround to the fintech not being able to dedicate engineering resources to send data to an API endpoint of ours. However, after a while, I thought to myself, there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously, and so now I'm curious to hear how common this practice is.

Maybe it's just me, but it feels like integrating with internal DBs could allow a startup to onboard and go live with customers much more quickly. For some reason though, I haven't heard much of this around, so to be completely honest, I'm thinking of starting a tool to make this process easier / standardised.

Before that though, I'm trying to wrap my head around the use cases for this so it'd be amazing to hear about your story of integrating directly with a customer's DB -- what was it for and how was the experience!

Re: Does anyone integrate with their customers' DB directly?

#2

Post by airpodsman »

I was thinking the same thing. I have an idea that I've been thinking about for some time but requires data to the Customer's Data. They'd have to expose us to ingest that either through : 1) an API that the customer exposes to you 2) Connected to the data source directly in a secure manner.

Option 1 seems like a lot of work for the customer ... probably not viable.

Option 2: you can securely connect to their data-source using oAuth2.0. In fact I know MongoDB offers this (https://www.mongodb.com/docs/atlas/security-oidc/) for this exact use-case I believe. I recall using Vercel to integrate with my MongoDB.

MongoDB: https://www.mongodb.com/docs/atlas/security-oidc/ Supabase: https://supabase.com/docs/guides/auth

Though I haven't done it directly, I think integrating with the DB through a secure protocol like oAuth is the way to go. DB services offer this and must be for this exact use case.

Re: Does anyone integrate with their customers' DB directly?

#3

Post by johnyeocx »

airpodsman wrote: post #2

I was thinking the same thing. I have an idea that I've been thinking about for some time but requires data to the Customer's Data. They'd have to expose us to ingest that either through : 1) an API that the customer exposes to you 2) Connected to the data source directly in a secure manner. Option 1 seems like a lot of work for the customer ... probably not viable. Option 2: you can securely connect to their data-so…

that's what I thinking actually, using oauth to connect to a DB is definitely a new concept though, and most larger enterprises either use data warehouses or self hosted DBs (just a guess), so oauth may not be an option

If the SaaS target customers are smaller companies / startups, then this may be viable

I'd love to hear more about your idea though! would you mind sharing a little bit about it?

Re: Does anyone integrate with their customers' DB directly?

#4

Post by drpossum »

> there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously

This is an unfortunate attitude.

From a software architecture point of view, you're hard-coupling your software to that database which can create brittleness: You're at the mercy of the original database schema which may not be optimal, carries it's own tech debt, may require translations to your own implementation, and may cause mistakes in interpretation in anything of scale. Even worse, you're at the customer's mercy if they want to change anything which immediately breaks your code (likely without warning) and now you have to fix it (likely under duress).

Letting an outside org get direct access to a database is an unnecessary security risk, even if you are "treating infosec seriously". I'd argue no organization that treats infosec seriously would want to request access to or grant access to anything more than the minimal information they need. https://en.wikipedia.org/wiki/Principle_of_least_privilege

Modern software design has found putting up abstractions and interfaces to address those issues which is why it's not particularly common. In fact, one fintech I worked with did this and ruined a team's holiday weekend by making an unfortunate and unsignaled change late on a Friday.

Re: Does anyone integrate with their customers' DB directly?

#5

Post by johnyeocx »

drpossum wrote: post #4

> there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously This is an unfortunate attitude. From a software architecture point of view, you're hard-coupling your software to that database which can create brittleness: You're at the mercy of the original database schema which may not be optimal, carries it's own tech debt, may require translations to your own implementation, and…

Thanks for sharing your views, these are definitely insightful and valid points. Infosec can never be overstated, and it was perhaps a bit naive of me to just think "treating it seriously" would be enough.

To argue against both the points you made though, there are reverse ETL platforms like Hightouch and Census today which 1. have specific architectures to make data movement from a central warehouse to third-party SaaS platforms seamless and easy, and 2. also access the company's data stores directly. What's the difference between what they're doing, and this hypothetical third-party unifying data integrations?

Re: Does anyone integrate with their customers' DB directly?

#6

Post by airpodsman »

johnyeocx wrote: post #3
airpodsman wrote: post #2

I was thinking the same thing. I have an idea that I've been thinking about for some time but requires data to the Customer's Data. They'd have to expose us to ingest that either through : 1) an API that the customer exposes to you 2) Connected to the data source directly in a secure manner. Option 1 seems like a lot of work for the customer ... probably not viable. Option 2: you can securely connect to their data-so…

that's what I thinking actually, using oauth to connect to a DB is definitely a new concept though, and most larger enterprises either use data warehouses or self hosted DBs (just a guess), so oauth may not be an option If the SaaS target customers are smaller companies / startups, then this may be viable I'd love to hear more about your idea though! would you mind sharing a little bit about it?

Let me know if you end up researching this further and what the conclusion is. I think the best way to tackle this would be via an API the customers can give you.

The idea is a simple marketing tool to help businesses reach out to customers that didn’t convert fully through the funnel. So for example, user created a profile, but didn’t end up purchasing.

If interested, I can share the link (once it’s up) and would love to get your feedback.

Re: Does anyone integrate with their customers' DB directly?

#7

Post by stackskipton »

I've been at companies that do this. It's extremely brittle process with significant overhead. Normally, we have to setup S2S VPN with all security/administrative implications of that. Then deal with any database schema changes and like.

I'd hope to never work at a company like that again as SRE, my life was nightmare.

Re: Does anyone integrate with their customers' DB directly?

#10

Post by Vampiero »

drpossum wrote: post #4

> there's nothing wrong with querying the fintech's DB as long as we're treating infosec seriously This is an unfortunate attitude. From a software architecture point of view, you're hard-coupling your software to that database which can create brittleness: You're at the mercy of the original database schema which may not be optimal, carries it's own tech debt, may require translations to your own implementation, and…

I would gladly take some brittleness over mountains and mountains of equally unmaintainable boilerplate and layers upon layers of indirection and abstractions... For what is essentially the same exact boring old CRUD that we've been doing for the past 40 years.

Who cares anyway? It's fintech. These projects are meant to become legacy and be replaced every 5-10 years. Those that don't are a PITA to work with regardless of how much the data layer is decoupled from the business layer.

And even if you do decouple them, these *** projects WILL find a way to break on a friday night anyway. Because of course they will. For the most part, they're written by juniors on tight deadlines and there's like only two people in the entire org who even have a faint clue of what the entire codebase does. You can use all the design principles and patterns you want, they're not going to save you.

Post reply on HN