Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How NoSQL forced the evolution of a scalable relational database

blog.memsql.com

Post reply on HN
Print view View original

201–210 of 211 posts

Re: How NoSQL forced the evolution of a scalable relational database

#201

Post by davidgould »

threeseed wrote: post #71
davidgould wrote: post #62

Earlier quoted context omitted.

I think you misunderstood the parents use of the term "model". All the things you describe here can be described in the relational model. The parent was arguing that the model is general enough to cover almost all collections of data. You appear to be arguing that for specific use cases specific implementations perform badly. These arguments pass each other unseen like ships in the night.

Actually wide tables for example can't be done at all. And technically we can use Microsoft Excel as a photo editor. But that's just ridiculous. Just like it's ridiculous to say you can use relational models for any data structure even if the queries will never return.

You are still missing the point. The relational model is not about particular implementations. There is no limit on number of columns etc. It is not "ridiculous to say you can use relational models for any data structure", it is the whole point. That particular implementations may not suite your requirements is not a problem of the model, it is a problem of the particular implementations.

Re: How NoSQL forced the evolution of a scalable relational database

#202

Post by dman »

jandrewrogers wrote: post #113
mmt wrote: post #81

Earlier quoted context omitted.

> It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job. But then it wouldn't be distributed processing! :) Seriously, though, consider it for a moment.. this pattern has similar features to something like Hadoop. The data comes from storage nodes (database server and, hopefully, their read replicas) and goes to processing nodes (ap…

Doing distributed joins correctly requires an architectural/technical capability that most distributed database engines don't have: decentralized parallel orchestration. If you have this, you can do joins even with very high data volumes efficiently given good parallel scheduling algorithms. Most databases are designed such that there is a single point of control that declaratively schedules all data flows required t…

Could you point out any open source supercomputing / data products that get this right (ie have decentralized data flows?).

Re: How NoSQL forced the evolution of a scalable relational database

#203

Post by TheCoelacanth »

lmm wrote: post #119
Amezarak wrote: post #93

Earlier quoted context omitted.

> Nobody is using joins at scale and every large company that started their scaling journey in a SQL database started by performing multiple queries on a distributed database. I've used joins "at scale" in multiple jobs in (conventional relational) databases up to a few TB with tens of thousands of transactions per second and never experienced any performance problem that was the result of a join, unless you count cr…

> never experienced any performance problem that was the result of a join, unless you count cross joins with no filters. An unindexed join will have major performance problems, and look exactly like an indexed join. > I have never seen an instance where doing a join was faster in code and I don't see how it could be except perhaps in some unusual edge cases. Faster to do the same thing? No. Easier to see which things…

SQL is also not very composeable. Queries and their constituent parts are not first class concepts in SQL. There is no way to, for instance, pass a query to a piece of code and have that code add part of a WHERE clause to that query. That results in a lot code dynamically generating SQL queries.

Re: How NoSQL forced the evolution of a scalable relational database

#204

Post by mmt »

dman wrote: post #202
jandrewrogers wrote: post #113

Earlier quoted context omitted.

Doing distributed joins correctly requires an architectural/technical capability that most distributed database engines don't have: decentralized parallel orchestration. If you have this, you can do joins even with very high data volumes efficiently given good parallel scheduling algorithms. Most databases are designed such that there is a single point of control that declaratively schedules all data flows required t…

Could you point out any open source supercomputing / data products that get this right (ie have decentralized data flows?).

I'm obviously not the parent commenter and have no inside knowledge of the HPC world, but my guess is the main open source supercompting projects would be from NASA, the US national labs, and CERN.

You might search HN for the recent announcements about new clusters, especially top500, and look for the comments discussing using MPI (versus something custom, I think?), as my recollection is that those topics would yield further pointers to the actual examples you're looking for.

Re: How NoSQL forced the evolution of a scalable relational database

#205

Post by hakfoo »

bunderbunder wrote: post #154
adrianpike wrote: post #131

Earlier quoted context omitted.

> I don’t understand the “call to get a quote” sales cycle. It's about maximizing revenue from enterprise clients - when the customer LTV is huge but the volume is low, it's smart to focus your resources on those dollar amounts. When the volume is high but the dollar amounts are low, it's smart to focus on optimizing the volume funnel.

More specifically, it's about price discrimination. If you publish a price, you need to publish a price that's going to be palatable to your smallest clients. But the moment you do so, your most deep-pocketed clients will demand the same price. This could be disastrous, because those same clients also tend to be more demanding, so a profitable price for one client could be a disastrous money-loser for another. If you…

If you're having to hide ALL pricing data, you're doing it wrong.

What's the difference between the small-business customer and the enterprise one? I'd expect it boils down to three big things:

* Quantities of resources, where you usually expect the per-unit price to go down for the big enterprise customer. The 5-million-request-per-month plan is usually cheaper per-request than the 100k-requests-per-month one. * SLAs, which can be done as tiered add-ons. * Support services tiers, which can be graduated sensibly.

I'll grant that there's a place for a "super-whale" tier where you're probably buying custom infrastructure solely to support that client, but there's room for that within a pricing page without scaring off a large number of customers. "For plans over n billion queries per month, please contact sales" means you can still make a reasonable direct sale for everyone else.

Even within the context of "we're a galactic enterprise ready to write a blank check", many firms will still want to buy a small off-the-shelf package as part of the validation process.

This could be to give the developers a sandbox to ensure it meets their expectations, or just to migrate a small portion of their systems before going head-first.

Re: How NoSQL forced the evolution of a scalable relational database

#206

Post by lotsofcows »

Sjenk wrote: post #84
lotsofcows wrote: post #72

Earlier quoted context omitted.

That's not a choice. To make a choice you need to be aware of alternatives. You use NOSQL because it was easy to achieve your goal.

Dont mean to troll. But how choosing for the easiest way to achieve your goal not a choice?

That's a different choice.

You might be "choosing" not to bother understanding the alternatives, but you're not making a choice between the alternatives because you don't know what they are.

Re: How NoSQL forced the evolution of a scalable relational database

#207

Post by infinite8s »

dman wrote: post #202
jandrewrogers wrote: post #113

Earlier quoted context omitted.

Doing distributed joins correctly requires an architectural/technical capability that most distributed database engines don't have: decentralized parallel orchestration. If you have this, you can do joins even with very high data volumes efficiently given good parallel scheduling algorithms. Most databases are designed such that there is a single point of control that declaratively schedules all data flows required t…

Could you point out any open source supercomputing / data products that get this right (ie have decentralized data flows?).

Take a look at parallel simulation codes (for example, LAMMPS in chemical physics).

Re: How NoSQL forced the evolution of a scalable relational database

#208

Post by richmarr »

threeseed wrote: post #51
richmarr wrote: post #45

Earlier quoted context omitted.

> This is basically a big, complicated, and slow piece of software that tries to make a SQL database into something else. Just to nitpick a little, but I think it's a worthwhile distinction, ORMs don't try to make a SQL database something else... either literally or philosophically. To use them in any more than a trivial way you still to understand RDBMSs. They just make the queries less verbose and the output more c…

> ORMs don't try to make a SQL database something else This is simply not true. Many ORMs are designed to abstract away SQL and RDBMS concepts entirely. They deal in objects and object graphs and output SQL which can be quite disjointed from the object model e.g. many-many relationships.

> This is simply not true.

Providing a wrapper around a thing is not the same as wanting to turn a thing into something else. I would argue that ORMs offer cosmetic change. Convenience, not substantive change.

> Many ORMs are designed to abstract away SQL and RDBMS concepts entirely.

Which ones? Look at their docs and you'll see where conditions, joins, columns, ordering, aggregations, etc.

ActiveRecord is probably the most abstract I've seen and the docs still refer endlessly to tables, foriegn keys, etc and show SQL equivalents http://guides.rubyonrails.org/active_record_basics.html#crea...

Here's Hibernate: http://docs.jboss.org/hibernate/orm/5.3/userguide/html_singl...

Here's Sequelize: http://sequelize.readthedocs.io/en/v3/docs/querying/#basics

Here's SQLAlchemy: https://docs.sqlalchemy.org/en/latest/orm/loading_relationsh...

I would say that ORMs as a class rely heavily on SQL and RDBMS concepts... particularly for anything beyond simple CRUD

Re: How NoSQL forced the evolution of a scalable relational database

#209

Post by ruiquelhas »

flavio81 wrote: post #124
189723954 wrote: post #38

Earlier quoted context omitted.

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a b…

>Everybody uses an ORM with a SQL database. That's not true. >While I was writing these queries, I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more" Well, no, because SQL is a 4th generation language of higher level than most general-purpose languages. It is a domain-specific language focused on database handling. A simple SELECT with a few joins and indexes involved e…

Why is Mongo a bad option for a document store? Serious question.

Re: How NoSQL forced the evolution of a scalable relational database

#210

Post by SkidanovAlex »

antirez wrote: post #31

Redis is not mentioned here at all so maybe the author is thinking mostly of other NoSQL software here, but well, in the case of Redis the whole point was not just the in-memory performance part, but the data model as well. My claim is that you can't really exploit the advantage of using memory if you perpetuate in using the memory to represent the same data model that you were using with relational databases. For in…

> To use SQL, even an in memory one, in such use case is not going to work.

Have an index on the score and select with ORDER BY ... LIMIT?

Post reply on HN