Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 264 posts

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#161

Post by orisho »

lonelappde wrote: post #142
krainboltgreene wrote: post #122

Earlier quoted context omitted.

Here I'll give you one: If you want to change a property of the database that will give your specific use improved performance, you have no way to transactionally apply that change. Rolling back becomes a problem of operational scale, rolling out as well. What if you need to release some feature, but that feature requires a database feature enabled? Normally you enable it once, in a transaction hopefully, and then ro…

You roll the feature out incrementally to the users who are ready. You build backwards compatible features. Basic software engineering.

Sure, this problem can be solved as can any other -- but there's a cost to it in development time. For every feature, the dev working on it had to do it in 3 steps to ensure backwards compatibility and handle partial migration failures gracefully. Imagine doing this for a small feature - the ops effort dwarfs the actual feature! Many small features would probably be dropped.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#162

Post by spicyramen »

My apps was a call center using Twilio. I did initially have a db for prototyping, then for demos, then we started growing and needed security and multi tenancy support. The pace and new customers started growing so fast that it just made sense to spin off new db instances (Postgres). We later started having issues updating schema, adding features, upgrading, troubleshooting. We needed to redesign the schema, backend and frontend but it was worth it at the end and saved us a lot of time.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#165

Post by gomox »

egamirorrim wrote: post #159
gomox wrote: post #119

Earlier quoted context omitted.

True, but: In my experience by the time you reach this point you have a lot of operational complexity because you and your team are used to your production cluster being a single behemoth, so chances are it's not easy to stand up a new one or the overhead for doing so is massive (i.e. your production system grew very complex because there is rarely if ever a need to stand up a new one). Additionally, a multi tenant b…

Some of the issues I see in one of my projects is high interactivity between accounts. E.g. if account 1 'sends' something to account 2 both of the shared/separate db instances need to be up or there'll need to be some kind of queueing mechanism. That's hard enough and then add to it that most clients want to BYOK to those instances

High interactivity between accounts is a good reason to not adopt the proposed multi-single-tenant architecture. The scenarios discussed are B2B enterprisey apps in which the environments are essentially independent from each other.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#166

Post by gomox »

mrunkel wrote: post #153
gomox wrote: post #149

Earlier quoted context omitted.

No. For example, HubSpot runs a multitenant system. URLs look like: https://app.hubspot.com/section/$ACCOUNT_ID/etc/etc In the simple, YAGNI implementation of this, when you create a new HubSpot account, most likely that will insert a new row into the accounts table, and the auto generated ID of that row will be your account ID. Therefore you need uniqueness to be enforced at that level. If you want to start running…

There are many ways to solve this that don't require uniqueness across all systems. https://${customer}.hubspot.com/.. . https://app.hubspot.com/${customer}/.. . You'd do this at the proxy/forwarder level.

Everything has a solution, but want to bet that at least 20 different internal systems at HubSpot assume that the account ID in that URL is globally unique?

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#167

Post by jokull »

At QuizUp, a mobile gaming startup, we did not have 1 per user but 1024 databases to be able to shard and scale. We needed pgbouncer to decrease connection counts and overhead. There was some extra pain, but managable. Postgres allows you to have this logical seperation on one database instance.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#168

Post by gomox »

orisho wrote: post #161
lonelappde wrote: post #142

Earlier quoted context omitted.

You roll the feature out incrementally to the users who are ready. You build backwards compatible features. Basic software engineering.

Sure, this problem can be solved as can any other -- but there's a cost to it in development time. For every feature, the dev working on it had to do it in 3 steps to ensure backwards compatibility and handle partial migration failures gracefully. Imagine doing this for a small feature - the ops effort dwarfs the actual feature! Many small features would probably be dropped.

Multi-single-tenant actually makes it easier to do transactional schema upgrades because these operations only impact one customer at a time. No fiddling with middle-of-the-migration state.

Re: Ask HN: Has anybody shipped a web app at scale with 1 DB per account?

#170

Post by wim »

We do this for our wiki/intranet SaaS app. In our case we don't use it for all data in the app though, but rather for a specific feature. One of the blocks customers can add to their wiki pages are form fields so they can build custom forms. The app then automatically generates a corresponding database in the backend to store records for their form, using a separate SQLite db.

Our original idea was to allow customers access to their form db directly so they could even run custom SQL queries on it and so on. In the end we actually never used that part (users just view and filter their records in the UI), so unless we still need this in the future, one could argue it's.. just slightly overengineered. It works well otherwise though, and migrations aren't really a problem for us, because they're done dynamically by the app (when people make changes to their form). The scaling is also no issue as one db will always be small enough to fit on one server, so we can "shard" easily by file. So I think the only obvious lesson here was make sure you really really need it (and if migrations/scaling would have meant a lot of overhead, I don't think we would have even considered it).

Post reply on HN