Does anyone integrate with their customers' DB directly? • Hacker News Board

Re: Does anyone integrate with their customers' DB directly?

#21

Post by cj » Tue, Sep 03, 2024, 12:37 AM UTC

johnyeocx wrote: post #3

airpodsman wrote: post #2

I was thinking the same thing. I have an idea that I've been thinking about for some time but requires data to the Customer's Data. They'd have to expose us to ingest that either through : 1) an API that the customer exposes to you 2) Connected to the data source directly in a secure manner. Option 1 seems like a lot of work for the customer ... probably not viable. Option 2: you can securely connect to their data-so…

that's what I thinking actually, using oauth to connect to a DB is definitely a new concept though, and most larger enterprises either use data warehouses or self hosted DBs (just a guess), so oauth may not be an option If the SaaS target customers are smaller companies / startups, then this may be viable I'd love to hear more about your idea though! would you mind sharing a little bit about it?

> If the SaaS target customers are smaller companies / startups, then this may be viable

I’m your target audience.

The #1 issue I have with this concept is, as a customer, I would have to trust that you understand the performance limits of my production database and that you won’t run (even accidentally) a burst of inefficient queries that takes down or impairs production.

One way around this would be to query against a replica / secondary, but even then you risk overloading a secondary/replica, which could have negative effects in a failover scenario.

Re: Does anyone integrate with their customers' DB directly?

#22

Post by deepsun » Tue, Sep 03, 2024, 12:46 AM UTC

tbrownaw wrote: post #17

Congratulations, schema changes are now directly visible outside your organization. For some of our outgoing files at $employer, there's a notice period of... I think I remember it being 30 days for additional code values and 90 days for layout changes. That sort of planning ahead becomes much harder if every single schema change is immediately visible to outsiders.

API is just another schema. You can make a VIEW in SQL db to keep that VIEW schema stable, while changing the source schema without downtime.

Re: Does anyone integrate with their customers' DB directly?

#23

Post by lloydatkinson » Tue, Sep 03, 2024, 12:46 AM UTC

thehappypm wrote: post #9

Data visualization products do this regularly

Are there any good open source/free ones? I tried Apache Superset a few months ago which seemed promising but was appalling bad when it came to trying to use it.

Re: Does anyone integrate with their customers' DB directly?

#24

Post by shermantanktop » Tue, Sep 03, 2024, 12:59 AM UTC

deepsun wrote: post #22

tbrownaw wrote: post #17

Congratulations, schema changes are now directly visible outside your organization. For some of our outgoing files at $employer, there's a notice period of... I think I remember it being 30 days for additional code values and 90 days for layout changes. That sort of planning ahead becomes much harder if every single schema change is immediately visible to outsiders.

API is just another schema. You can make a VIEW in SQL db to keep that VIEW schema stable, while changing the source schema without downtime.

That’s great for making db operations easier. Just because you make a pseudo-API out of a view doesn’t mean you should.

The problems are in the realm of non-functional requirements. Quotas, security, protocol support, asynchronous batch and bulk data access, granular access control, throttling…you can try to do this with your db, and many have, but the square peg will eventually not fit anymore.

Re: Does anyone integrate with their customers' DB directly?

#25

Post by anitil » Tue, Sep 03, 2024, 1:00 AM UTC

I've had to do this due to tech not getting a say in vendor choice. We were given an SSH tunnel, a DB user and a query to run by the vendor and sent on our way. Of course, periodically that query stops working because 'woops we changed our schema', and we have to wait for a new query for us to run.

In terms of Pros: - It was quick to set up and _seems_ to work, mostly

For Cons: - We have no guarantees that this query will continue to work - We have no understanding of the model behind this data (why are these records in this state, and these in another? No idea, and the vendor doesn't seem to understand the question) - We need to poll this data source. How often? No idea!

Culturally I think it suggests an immature tech organisation in general, so while not a red flag I'd suggest it is an orange flag.

Re: Does anyone integrate with their customers' DB directly?

#26

Post by 486sx33 » Tue, Sep 03, 2024, 1:06 AM UTC

Trigger my database and you’re fired, software scab!

But in reality, any big corp assumes if you have the access, you’ve used it. So it should be pretty common

Re: Does anyone integrate with their customers' DB directly?

#27

Post by segmondy » Tue, Sep 03, 2024, 1:14 AM UTC

I've had companies ask to read it DB, answer is no and over my dead body. They get API access or data dump. I've been asked to read external companies DB, I always refuse, give me API or data dump. I don't even agree for internal teams to share DBs. It's a bad idea, but I can see the draw for startups and small companies without the skills. BTW, can we chat about your former charge acts oriented startup? My info is in my profile.

Re: Does anyone integrate with their customers' DB directly?

#28

Post by RajT88 » Tue, Sep 03, 2024, 1:16 AM UTC

Not me, but have seen this with a number of vendors.

- Customer buys a service from a vendor

- Customer creates a Vnet peering in Azure between their subscriptions

- Customer queries the vendor's DB's directly

I will not name names for the vendors or customers, but you have heard of all of them.

Re: Does anyone integrate with their customers' DB directly?

#29

Post by rco8786 » Tue, Sep 03, 2024, 1:16 AM UTC

> the fintech not being able to dedicate engineering resources to send data to an API endpoint of ours

There's an extremely real chance that this fintech giving you direct DB access was breaking the law by doing so.

Re: Does anyone integrate with their customers' DB directly?

#30

Post by mr_toad » Tue, Sep 03, 2024, 1:20 AM UTC

Many DBAs don’t even like internal analysts directly querying production line of business databases. Usually they’d set up replicas, or even a full blown data warehouse.