Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Relational databases aren’t dinosaurs, they’re sharks

simplethread.com

Post reply on HN
Print view View original

81–90 of 135 posts

Re: Relational databases aren’t dinosaurs, they’re sharks

#81

Post by bob1029 »

aenis wrote: post #26

Another consideration is that, at scale, no sql is way cheaper. I run a service with approx. 900k daily users, each generating about 210 object writes and reads that need to execute within 50ms, and I am running this on firestore for about $350 a month, incl. Elb, waf, regionally replicated compute, managed NLP and translate. I sync the no sql stuff to bigquery for analytical usage. Cheap, and scales without any prob…

> Another consideration is that, at scale, no sql is way cheaper.

Define "scale" and "cheaper".

I have some SQLite databases that are chewing through over 1K writes per second in production (WAL=on). Many of our transactions resolve within 10-100 microseconds because the application process and the database process are the same thing.

SQLite seems to be one of the cheapest options available, being totally free and of negligible footprint. It's also exceedingly cheap in terms of engineering complexity and scary unknowns. Something about being literally the most deployed piece of software on earth seems to help.

NoSQL reads like a religion to me these days. Starting with something that is actively trying to tear itself apart due to the inherent complexity seems like a horrible way to build a 5 user POC.

Re: Relational databases aren’t dinosaurs, they’re sharks

#82

Post by chousuke »

cageface wrote: post #17
laurent92 wrote: post #7

For some reason, Java developers didn’t like writing SQL, so we introduced Hibernate which “does SQL for you”. Hibernate creates appallingly bad SQL, so “databases are slow”. Particularly when using a getter on a lazy-loaded relationship. A query might end up taking 1ms per record instead of 10ms for 10k records. You can rewrite all you want in Hibernate and greatly improve performance, but you often need to introduc…

Building SQL queries by gluing strings together is tedious and error prone. So you write a lot of helper functions to build queries for you and pretty soon you've invented your own crappy ORM. Why not save a lot of time and bugs and use a battle tested ORM and drill down to SQL for the queries that really matter instead?

There's a difference between an ORM and a query builder. Full-blown ORMs often allow you to pretend that all you have are "regular" objects that just happen to do database queries when you access them; and that's the kind of ORM that often leads to inefficient database access and fragile code when the program throws an exception when you just wanted to iterate over a list because that iteration actually caused a fetch from a database that happened to have a failover event.

With a query builder, you can still use the database in a type-safe way and without gluing strings together, but accessing the database is also more explicit, so mistakes are harder to make.

It's fine to represent the result of a database query as a sequence, but its type and interface should be different from that of eg. a list datastructure, because it can fail. similarly, if you have "User" objects representing users, it needs to be explicit which methods fetch data from the database and which use only in-memory properties that code may freely access. Preferably, have different types for "UserInDB" and "User" altogether, with some glue to go between them.

Re: Relational databases aren’t dinosaurs, they’re sharks

#83

Post by jcpst »

vharuck wrote: post #65
squeaky-clean wrote: post #58

Earlier quoted context omitted.

I work with locations and addresses on an international system. They are not anywhere near as standardized as to allow this. And also at scale, you'll get things that seem like they shouldn't be addresses but are. One of our addresses that caused trouble is literally: "The yellow sign across the street from the Seven-Eleven at ". We have one address that's legally in two countries at the same time. One address is jus…

I've come around to this view, too. Also like names, the best solution is to avoid using them for analysis. Run them through an address geocoding service and store the coordinates next to the address. Use the original address for sending mail or filling out forms, and coordinates for analysis. Unless cleaning, parsing, and geocoding addresses is one of your core business values, let somebody else do it. It's a lot of…

I agree. We’ve built automation that ingests geocoding data from multiple services, and keeps the system up to date. The models for an “Address“ get complex, a single entity can have many different types of addresses.

Re: Relational databases aren’t dinosaurs, they’re sharks

#84

Post by rendall »

chrisandchris wrote: post #14
pier25 wrote: post #2

The term NoSQL is meaningless. It only means that a database is not SQL (d'oh) but people (such as the author of the article) use it as if it meant anything beyond that. Talking about "NoSQL tradeoffs" implies all non SQL databases share similar features, operational models, use cases, etc, which is simply not true. For example, DynamoDB, Mongo, and Fauna have absolutely nothing in common.

> The term NoSQL is meaningless. It only means that a database is not SQL (d'oh) but people (such as the author of the article) use it as if it meant anything beyond that. […] Except that it does not. It does mean „Not Only SQL“, and not „no SQL“.

@chrisandchris is correct. NoSQL means "Not Only SQL"

Furthermore, the article is uninformed and writes as if "NoSQL" is an alternative paradigm to SQL. In fact, NoSQL covers a whole range of paradigms and approaches, from key-value, to document, to graph, to more exotic flavors. Some of which can even be queried with SQL

ACID can be a feature of other database paradigms as well, if necessary. With MongoDB Atlas, for instance, an engineer can ensure that data consistency is high priority across clusters. Or not, if that's not important.

On top of all that, table-based database management systems are designed to prioritize saving hard drive space over cpu cycles. As cpu cycles have become more expensive relative to "hard drive space", the need for this kind of database has declined.

Re: Relational databases aren’t dinosaurs, they’re sharks

#86

Post by tharkun__ »

j16sdiz wrote: post #51

I don’t understand why we still need these kind of article.. After all these years, I thought the advantage and trade-off of different database should be well-understood. But the fact is, there are still lots of mis-infomation floating around. It looks like the lesson we have learnt are not communicated to border groups of engineers.

I've made this comment a few times on HN, one very recently - so apologies if someone reads it twice now - but your comment really wants to make me do it again.

When I talked to my dad about RDBMS he was like "weeeelll, sure, there's things like DB2 UDB that do relational but performance wise, nothing beats reading the data straight by key in exactly the format you need.".

DB2 UDB: "Initial release: 1987; 34 years ago". I.e. what he would rather use and is sort of the NoSQL equivalent is _even older than that_. DBM (Ken Thompson - released by AT&T in 1979) comes to mind, tho I don't remember exactly what it was he was using/referring to, which would have been something that would rather run on S/360 and S/370 systems. It's been a while.

Background: he started off with 360 assembler and worked all his working life on IBM Mainframes and the various technologies in and around it. They had it all and they had it before it came to "us". We're just re-inventing most of these things on much cheaper and more open hardware and software.

All that to say: We still need these kinds of articles, because people "tend to forget". Or not even check "prior art". There was a recent article and comments around even research papers essentially being re-done and presented as novel research. And on some level that is even correct, because the authors genuinely came up with the same ideas and research as the original authors did. But 20+ years after the fact.

Re: Relational databases aren’t dinosaurs, they’re sharks

#87

Post by zepolen »

HWR_14 wrote: post #69
zepolen wrote: post #50

Earlier quoted context omitted.

That's more the benefits of cloud rather than nosql though. 200 writes/second is not even scale, a single instance would handle that fine, with beefy hardware you could do 100x that. You could even run it in cloud, for about $50/month, and as a bonus use the same instance to perform the analytics which will be real time. Not to mention enjoying much better data integrity.

Which cloud lets you rent a beefy SQL server for $50/mo? This is actually really useful info to me, as we're looking at needing to migrate to a new provider soon.

I mean you're going to have to be more specific than that, some clouds have high cpu but expensive disk, others the reverse. $50/mo will not get you both, but it will easily get you 200tx/s.

Re: Relational databases aren’t dinosaurs, they’re sharks

#88

Post by wbl »

jhgb wrote: post #73
avianlyric wrote: post #70

Earlier quoted context omitted.

> I understand these issues. I seriously doubt you do. The closest you can get to “standard address format” is: Address Line 1 Address Line 2 Address Line n Postal code (which can be blank) Postal Area (which can be blank) Country There is no way to build “first-class” support for addresses. Because theres no such thing as a valid or invalid address, only whether or not someone can find the correct location by readin…

I don't see how this changes anything. You can accommodate any address format, or any finite union of multiple address formats, including any computed or materialized views of the address with a custom type (AND including an "I give up" default for when everything else fails). > there’s no standard, it’s entirely dictated by local conventions, which can change street-to-street, city-to-city Well, that may be an inter…

I know what to write on my envelopes so they get to me. You do not. Your job is to make it possible for things you send to get to me, not the other 3 units in the apartment.

Good luck with the Oakland addresses in Berkeley.

Re: Relational databases aren’t dinosaurs, they’re sharks

#89

Post by pier25 »

pbourke wrote: post #6
pier25 wrote: post #2

The term NoSQL is meaningless. It only means that a database is not SQL (d'oh) but people (such as the author of the article) use it as if it meant anything beyond that. Talking about "NoSQL tradeoffs" implies all non SQL databases share similar features, operational models, use cases, etc, which is simply not true. For example, DynamoDB, Mongo, and Fauna have absolutely nothing in common.

At this point we all generally understand that NoSQL means a system that lacks one or more of: the SQL language, a relational model or ACID guarantees. It’s a useful shorthand for all of that.

If that's the case, it wouldn't make sense to talk about NoSQL tradeoffs as if all NoSQL dbs were actually similar in any way.

Eg:

Fauna is considered a NoSQL database and doesn't have any of the drawbacks the article mentions. It has ACID guarantees, a relational model, and strong consistency.

Mongo and Dynamo also offer transactions with ACID guarantees these days.

Etc.

Re: Relational databases aren’t dinosaurs, they’re sharks

#90

Post by jhgb »

wbl wrote: post #88
jhgb wrote: post #73

Earlier quoted context omitted.

I don't see how this changes anything. You can accommodate any address format, or any finite union of multiple address formats, including any computed or materialized views of the address with a custom type (AND including an "I give up" default for when everything else fails). > there’s no standard, it’s entirely dictated by local conventions, which can change street-to-street, city-to-city Well, that may be an inter…

I know what to write on my envelopes so they get to me. You do not. Your job is to make it possible for things you send to get to me, not the other 3 units in the apartment. Good luck with the Oakland addresses in Berkeley.

Yes, I said that the complete type would have to have components developed by others as well, since I can't provide input for other countries. Doesn't mean that a minimum complexity formalization is impossible (in fact, mathematically, by enumeration one such formalization must exist). An "as-given" component is obviously always going to work for you.
Post reply on HN