Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Relational databases aren’t dinosaurs, they’re sharks

simplethread.com

Post reply on HN
Print view View original

41–50 of 135 posts

Re: Relational databases aren’t dinosaurs, they’re sharks

#41

Post by lmilcin »

The choice of technology by developers and their managers is guided in large part by trends rather than by wholly rational decision process.

The type of database should be dictated by the application requirements.

Relational databases provide some exceptional guarantees while also being able to run quite large systems.

This means, when you have:

1. Relational data,

2. Queries that are not known beforehand,

3. Data that can fit one server or can be sharded to fit,

that RDBMS is probably the best choice for you.

You may not like SQL as a language but at least there is large body of knowledge on how to use SQL effectively for your problem, how different choices affect performance, etc. And a lot of very good tools to help you with that.

I have seen time and time again small teams to "revolt" against SQL databases choosing something like Cassandra or MongoDB. The effect that the team spends now years learning the new database, complicates their application to provide same functionality they got from SQL for free, contorts the data to the new paradigm.

My team chose, years ago, before I came, to use MongoDB for what is very relational problem. This resulted in huge duplication, performance issues and complexity on the application side.

No, the team does no longer have SQL problems. Instead we have other problems that consume large part of our focus, rather than use it to make the product better.

Re: Relational databases aren’t dinosaurs, they’re sharks

#42

Post by the-dude »

thatwasunusual wrote: post #11
jseban wrote: post #10

Earlier quoted context omitted.

I think your solution here was to move the typing to the application layer, which kind of makes sense because that's where you know the locale. But why also move the data to NoSQL, I don't see what that would add. If you had already removed the typing from the relation DB I think that would have worked as well, or am I missing something?

The main problem was that we were unable to store the different types of addresses because of their different "layouts" in a "relational way."

I understand it might not be possible due to other contraints, but in smaller projects with international addresses I just used a text/memo field for the address and seperate field for the country.

Addresses now can be entered the way the user prefers.

Re: Relational databases aren’t dinosaurs, they’re sharks

#43

Post by gadrev »

hnfong wrote: post #39
thatwasunusual wrote: post #11

Earlier quoted context omitted.

The main problem was that we were unable to store the different types of addresses because of their different "layouts" in a "relational way."

I don't quite understand "was before RDBMS solutions supported JSON "... did (for example) MySQL not support storing JSONs as binary blobs or text before?

You could always store JSON as text, but you'd miss out on the ability to query over JSON efficiently, among some other things, see [1]. Maybe that's what made it useful/viable to GP.

[1] https://dev.mysql.com/doc/refman/5.7/en/json.html

Re: Relational databases aren’t dinosaurs, they’re sharks

#44

Post by nimchimpsky »

jhgb wrote: post #31
thatwasunusual wrote: post #8

Great article, and I this particularly brought back memories for me: > [...] in some instances you might work with vast quantities of data, or deal with transactional systems that just don’t easily fit the operational limitations of relational databases. And in those cases, you should consider moving some, or all, of your data into a non-relational database. I've worked with a big application that utilised this appro…

> Our solution was to move the typing of this information to the application layer, and just store the arbitrary data in a NoSQL solution. This worked perfectly, and to my knowledge it's still working without a glitch. > This was before RDBMS solutions supported JSON, and if I were to do this again, I'd probably just continue to use MySQL, PostgreSQL or whatever, and store it as JSON in the database. Both solutions s…

you want to store addresses using the geo coordinates ? Thats the worst idea I've heard in a while.

Re: Relational databases aren’t dinosaurs, they’re sharks

#45

Post by kaba0 »

laurent92 wrote: post #7

For some reason, Java developers didn’t like writing SQL, so we introduced Hibernate which “does SQL for you”. Hibernate creates appallingly bad SQL, so “databases are slow”. Particularly when using a getter on a lazy-loaded relationship. A query might end up taking 1ms per record instead of 10ms for 10k records. You can rewrite all you want in Hibernate and greatly improve performance, but you often need to introduc…

ORMs are made primarily for OLTP workloads, and they are the primary means of converting SQL results into objects.

There is nothing inherently slow about Hibernate (other than people not knowing how to use a tool and blaming it), and frankly, providing a unified SQL dialect that ports over another DB is a plus.

Re: Relational databases aren’t dinosaurs, they’re sharks

#46

Post by makapuf »

kroolik wrote: post #28
ransom1538 wrote: post #22

Earlier quoted context omitted.

IMHO Blobs usually turn into a mess though. Some VP will ask "what weekday do users comment the most". With blobs this is drama. You end up parsing all these blobs and creating relational tables anyway. Sure you could do metrics separately -- but good luck predicting what stats you will want in the future. Need an admin portal? With blobs you are coding all these custom admin pages. With mysql, I love when new people…

> Some VP will ask "what weekday do users comment the most". This is why people invented read models. Because the data model that works for the app and makes it easy to work with there might not necessarily work for analytics. Generate analytics data from you app model so that both world can move independently. > Need an admin portal? With blobs you are coding all these custom admin pages. Admin portal with direct ac…

or logic is in db, with validation triggers and constraints constraints, but that is not always the case.

Re: Relational databases aren’t dinosaurs, they’re sharks

#47

Post by cpursley »

jhgb wrote: post #35
cpursley wrote: post #20

Earlier quoted context omitted.

This is exactly what I do - store location components as jsonb in Postgres

Why can't you make a proper, first-class supported custom type in your database instead?

I tried at first, google location data is somewhat inconsistent. I do pull out some of the items and stick them in a varchar column. And I also pull out and store long/lat as a postgis type (which is an amazing bit of tech btw).

Re: Relational databases aren’t dinosaurs, they’re sharks

#48

Post by nicoburns »

ransom1538 wrote: post #22
cpursley wrote: post #20

Earlier quoted context omitted.

This is exactly what I do - store location components as jsonb in Postgres

IMHO Blobs usually turn into a mess though. Some VP will ask "what weekday do users comment the most". With blobs this is drama. You end up parsing all these blobs and creating relational tables anyway. Sure you could do metrics separately -- but good luck predicting what stats you will want in the future. Need an admin portal? With blobs you are coding all these custom admin pages. With mysql, I love when new people…

With Postgres’s JSONB type you can query and index the unstructured JSON data just like regular columns. You generally want to avoid storing arrays of data in JSONB as much as possible, but unstructured objects are just as easy to query as regular columns with only a small performance hit.

Re: Relational databases aren’t dinosaurs, they’re sharks

#49

Post by PedroBatista »

aenis wrote: post #26

Another consideration is that, at scale, no sql is way cheaper. I run a service with approx. 900k daily users, each generating about 210 object writes and reads that need to execute within 50ms, and I am running this on firestore for about $350 a month, incl. Elb, waf, regionally replicated compute, managed NLP and translate. I sync the no sql stuff to bigquery for analytical usage. Cheap, and scales without any prob…

You’re comparing a technology with a product. Would it be $350 a month if you had to run all that by yourself?

You’re enjoying vast economies of scale and for practical terms it’s fine. For now all those bits with cute names aren’t dinosaurs but for damn sure they aren’t sharks.

Re: Relational databases aren’t dinosaurs, they’re sharks

#50

Post by zepolen »

aenis wrote: post #26

Another consideration is that, at scale, no sql is way cheaper. I run a service with approx. 900k daily users, each generating about 210 object writes and reads that need to execute within 50ms, and I am running this on firestore for about $350 a month, incl. Elb, waf, regionally replicated compute, managed NLP and translate. I sync the no sql stuff to bigquery for analytical usage. Cheap, and scales without any prob…

That's more the benefits of cloud rather than nosql though.

200 writes/second is not even scale, a single instance would handle that fine, with beefy hardware you could do 100x that.

You could even run it in cloud, for about $50/month, and as a bonus use the same instance to perform the analytics which will be real time.

Not to mention enjoying much better data integrity.

Post reply on HN