Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Startup Mistakes: Choice of Datastore

stavros.io

Post reply on HN
Print view View original

111–119 of 119 posts

Re: Startup Mistakes: Choice of Datastore

#111

Post by StavrosK »

_m8fo wrote: post #110
StavrosK wrote: post #53

Earlier quoted context omitted.

If you want to create offline-first apps, I would use Postgres as the main datastore and use Couch to sync data between client and server. You'd have to decide which of your data would live where (or if you wanted to use Couch as a way to transfer data from the server to the client). Couch is a very good datastore for that use case, though, so I would definitely use it in some capacity for your purpose.

Can you expand on this? Do you mean you'd use Postgres as your main store and periodically "flush" data from CouchDB to Postgres (probably as JSON)? I don't see how that would work reliably. On the client would you source CouchDB or Postgres? Presumably you'd access CouchDB directly, but then why even use Postgres (for that subset of data, anyway).

It really depends on what your data looks like. If it's just game settings and state, put them in Couch and that's it, and keep user data like payments and activity there.

If it's data you're going to want to run analytics on and sync to the client, you're probably going to have to store it in both places, I think.

Re: Startup Mistakes: Choice of Datastore

#112

Post by erik_seaberg »

alkonaut wrote: post #91
maephisto wrote: post #18

Earlier quoted context omitted.

MongoDB/NoSQL is deployed and used at large scale by companies like Facebook, Ebay and many others. That's quite far from "no one is actually making that choice nowadays".

> companies like Facebook, Ebay They didn't chose NoSQL, they were forced to. I'm fairly convinced they started with relational stores. If a company or product grows to a point where relational data doesn't work, that's a problem you want to have. The mistake is either thinking you need to design for facebook scale from the beginning OR thinknig that you can cut time in a startup by not having to bother with those pe…

Facebook started with normalized tables, though they altered the schema really frequently. They added more MySQL servers as they grew to more schools. Then their users started graduating and moving around and things got complicated.

Re: Startup Mistakes: Choice of Datastore

#113

Post by logicallee »

StavrosK wrote: post #108
logicallee wrote: post #104

Earlier quoted context omitted.

That's fair. As an aside, while it solved your crash issue and obviously scales, when you switched did you get an immediate measurable (and noticeable) performance hit due to the overhead of a proper concurrent database?

No, no noticeable hit. SQLite is a proper concurrent database as well, so I don't think I would have.

I don't mean to be dense but why did you just say "SQLite is a proper concurrent database as well" after sharing a story to the contrary and advising "Don't use SQLite for production".

I'm just trying to understand your advice and learn from your experience, I am just confused by this followup.

Re: Startup Mistakes: Choice of Datastore

#114

Post by StavrosK »

logicallee wrote: post #113
StavrosK wrote: post #108

Earlier quoted context omitted.

No, no noticeable hit. SQLite is a proper concurrent database as well, so I don't think I would have.

I don't mean to be dense but why did you just say "SQLite is a proper concurrent database as well" after sharing a story to the contrary and advising "Don't use SQLite for production". I'm just trying to understand your advice and learn from your experience, I am just confused by this followup.

It's not an issue with SQLite itself, the python bindings don't work for concurrent access (and I think some other languages' bindings as well). If you're going to embed SQLite yourself, I think it works fine (I haven't tried it).

In theory, SQLite works quite well with concurrent accesses, I have just never tried it because the libraries in the languages I've used didn't work well for that.

Re: Startup Mistakes: Choice of Datastore

#115

Post by logicallee »

StavrosK wrote: post #114
logicallee wrote: post #113

Earlier quoted context omitted.

I don't mean to be dense but why did you just say "SQLite is a proper concurrent database as well" after sharing a story to the contrary and advising "Don't use SQLite for production". I'm just trying to understand your advice and learn from your experience, I am just confused by this followup.

It's not an issue with SQLite itself, the python bindings don't work for concurrent access (and I think some other languages' bindings as well). If you're going to embed SQLite yourself, I think it works fine (I haven't tried it). In theory, SQLite works quite well with concurrent accesses, I have just never tried it because the libraries in the languages I've used didn't work well for that.

ohhhhh, I get it. Yes, you mentioned this in your original comment ("the client libraries are usually locking accesses") but I guess I read too quickly.

Though unless you did a truly comprehensive shootout it might be fairer to write: "Be careful using sqlite in production, the client libraries I tried in Python did not handle concurrent reads/writes, whereas clients for postgresql handle it just fine."

Anyway, thanks for the clarification!

Re: Startup Mistakes: Choice of Datastore

#116

Post by dordoka »

codazoda wrote: post #52
dordoka wrote: post #42

Earlier quoted context omitted.

What's the problem with a .io domain? Honest question.

There was a recent failure that affected everyone using the TLD and a popular blog post explaining it and suggesting that you use something more reliable. Some of the comments on the article pointed out that the same company runs .org and a handful of other reliable TLD's.

Thanks very much

Re: Startup Mistakes: Choice of Datastore

#117

Post by dang »

mamcx wrote: post #16
onion2k wrote: post #2

Just pick something and build an MVP with it. Then get on with the hard part of finding paying customers. You can fix the bad tech decisions later. Without customers it won't matter which database you used before your startup failed.

BIG ERROR. Like, MONUMENTAL. That something (if chose very wrong) will totally derail you progress and will cost a lot of fix it later. Is incredible. Nobody remember that the most cost effective way to fix a problem is in the early stages? And, yes, the best overall primary datastore, like 90% of the cases, is a RDBMS. Very few actually need to use something else.

Could you please not use uppercase for emphasis? It's basically yelling. Asking not to do that is one of the original HN guidelines: https://news.ycombinator.com/newsguidelines.html. Imagine this comment in all caps and I'm sure you'll understand why.

Re: Startup Mistakes: Choice of Datastore

#118

Post by hvidgaard »

jrobn wrote: post #25
onion2k wrote: post #2

Just pick something and build an MVP with it. Then get on with the hard part of finding paying customers. You can fix the bad tech decisions later. Without customers it won't matter which database you used before your startup failed.

I would suggest that something be PostgreSQL. With JSONB and and all the other goodies in version 10 they are too numerous to list there is no reason not to use it for 90% of apps. It WILL scale until you need something more specialized. Postgres is far from a boring relational database. Don’t make the mistake of building a mvp on rickety stilts and then swap them out later for proper columns while trying to run.

Postgre will be good for 99.99% of all workloads. And when you need something else, you should have enough revenue to migrate the parts that make sense to a different datastore.

Re: Startup Mistakes: Choice of Datastore

#119

Post by jarym »

I think the real mistake is one of: 1. I'm gonna use NoSQL because I've heard its really cool. 2. I'm not going to use SQL because its hard for me to write queries and change stuff on the fly.

If either of those are true (gotta be honest with yourself when deciding) then STOP. You have to think about the best tool for the job overall - sometimes that will be NoSQL, sometimes it will be an RDBMS. If you can't decide which of the two, then Postgres with its JSON support is IMO the best starting point.

Post reply on HN