Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

news.ycombinator.com

Post reply on HN
Print view View original

151–160 of 264 posts

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#151

Post by sah2ed »

lonelappde wrote: post #142
krainboltgreene wrote: post #122

Earlier quoted context omitted.

Here I'll give you one: If you want to change a property of the database that will give your specific use improved performance, you have no way to transactionally apply that change. Rolling back becomes a problem of operational scale, rolling out as well. What if you need to release some feature, but that feature requires a database feature enabled? Normally you enable it once, in a transaction hopefully, and then ro…

You roll the feature out incrementally to the users who are ready. You build backwards compatible features. Basic software engineering.

> You roll the feature out incrementally to the users who are ready. You build backwards compatible features. Basic software engineering.

The parent mentioned having to support ~150 B2B customers, so the effort is amplified x100 — more than 100 individual customers databases have to be coddled as you’ve described, albeit they are stuck with poor tooling to manage changes across their unusual architecture.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#152

Post by paxys »

You don't have to pick one or the other. We have a standard horizontally sharded database setup where large customers get their own servers while smaller ones are colocated.

One thing we do strictly enforce is that the schema of the entire database must be consistent, so no one-off changes or migrations per customer. Database partitioning is completely opaque from the application's perspective.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#153

Post by mrunkel »

gomox wrote: post #149
ehsanu1 wrote: post #143

Earlier quoted context omitted.

I think you are talking about problems going from multiple single-tenant systems to a single multi-tenant system. You parent is talking about the opposite.

No. For example, HubSpot runs a multitenant system. URLs look like: https://app.hubspot.com/section/$ACCOUNT_ID/etc/etc In the simple, YAGNI implementation of this, when you create a new HubSpot account, most likely that will insert a new row into the accounts table, and the auto generated ID of that row will be your account ID. Therefore you need uniqueness to be enforced at that level. If you want to start running…

There are many ways to solve this that don't require uniqueness across all systems.

https://${customer}.hubspot.com/... https://app.hubspot.com/${customer}/...

You'd do this at the proxy/forwarder level.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#155

Post by tobyhede »

Years ago I worked for a startup that provided CMS and ecommerce software for small business. Each of our 3000+ customers had their own MySQL database.

We had a long tail of customers with negligible usage and would run several thousand MySQL databases on a single server. As customers scaled we could migrate the database to balance capacity. We could also optionally offer "premium" and "enterprise" services that guaranteed isolation and higher durability.

Scaling was never a real issue, but the nature of our clients was steady incremental growth. I don't think we ever had a case of real "overnight success" where a shared host customer suddenly melted the infrastructure for everyone.

However, managing and migrating the databases could be a real issue. We had a few ways of handling it, but often would need to handle it in the code, `if schemaVersion == 1 else`. Over time this added up and required discipline to ensure migration, deprecation and cleanuop. As a startup, we mostly didn't have that discipline and we did have a fair bit of drift in versions and old code lying around.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#156

Post by qaq »

We did this for 2 large projects I worked on. Works really well for env. where you can get a lot of data per customer. We had customers with up to 3-4 TB databases so any other option would either be crazy expensive to run and or to develop for. You need to invest a bit of time into nice tooling for this but in a grand scheme of things it's pretty easy to do.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#157

Post by paulmendoza »

I have used this architecture at 2 companies and it is by far the best for B2B scenarios where there could be large amounts of data for a single customer.

It is great for data isolation, scaling data across servers, deleting customers when they leave easily.

The only trick are schema migrations. Just make sure you apply migration scripts to databases in an automated way. We use a tool called DbUp. Do not try to use something like a schema compare tool for releases.

I have managed more than 1500 databases and it is very simple.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#158

Post by sah2ed »

fergie wrote: post #148

My memory might be a bit shaky on this, but I am pretty sure that Facebook was running "one solr instance per user" in the early days and that (Apache) Cassandra was developed from this idea. Generally any large social network will need to follow the 1 database per account strategy to some extent because of the tricky many-to-many relationship that groups/users have with other groups/users- this creates a cartesian p…

> Generally any large social network will need to follow the 1 database per account strategy to some extent because of the tricky many-to-many relationship that groups/users have with other groups/users- this creates a cartesian product that is too large to traverse in reasonable time with one large database.

Once cartesian products start to reach certain sizes, a few trade-offs can be made.

The compute cost of traversing complex entity relationships in a relational database can be mitigated by using denormalization—allowing lots of duplicates since storage is cheap.

Another alternative is to switch to a network database so entity relationships can be traversed using graph semantics.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#159

Post by egamirorrim »

gomox wrote: post #119
cornishpixels wrote: post #118

Earlier quoted context omitted.

You can always take a multi-tenant system and convert it into a single-tenant system a lot more easily. First and foremost, you can simply run the full multi-tenant system with only a single tenant, which if nothing else enables progressive development (you can slowly remove those now-unnecessary WHERE clauses, etc).

True, but: In my experience by the time you reach this point you have a lot of operational complexity because you and your team are used to your production cluster being a single behemoth, so chances are it's not easy to stand up a new one or the overhead for doing so is massive (i.e. your production system grew very complex because there is rarely if ever a need to stand up a new one). Additionally, a multi tenant b…

Some of the issues I see in one of my projects is high interactivity between accounts. E.g. if account 1 'sends' something to account 2 both of the shared/separate db instances need to be up or there'll need to be some kind of queueing mechanism.

That's hard enough and then add to it that most clients want to BYOK to those instances

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#160

Post by Polylactic_acid »

swlkr wrote: post #137

I worked for a company that did this, we had hundreds of database instances, one per customer (which was then used by each of those customers' employees). It worked out pretty well. The only downside was that analytics/cross customer stats were kind of a pain. The customers all seemed to like that their data was separate from everyone else's. This never happened, but if one database was compromised, everyone else's w…

It has been an actual requirement from our customers that they don't share an instance or database with other customers. It also seriously limits the scope of bugs in permissions checks. Sometimes I will find a bit of code that should be doing a permissions check but isnt which would be a much bigger problem if it was shared with other companies.
Post reply on HN