101–110 of 211 posts
Well, this is exactly why competition is good, in every domain. That's why PostgreSQL got its column mode and JSON type, because there was a clear use case for performance for the former and schema-less data for the latter. EAV pattern is a plague and I'm glad documents are replacing it.
Why EAV is plague? Isn't JSON type a wrapper of EAV?
Earlier quoted context omitted.
>I've never understood the "joins are slow" meme or where it came from. Well SQL databases generally don't support joining across a sharded database, which is usually necessary to scale unless you try to scale vertically with high powered machines and your data fits into memory and so on. They are also obviously slow compared to denormalizing and querying without a join. Then there is the other fact I mentioned that…
> Yes. A programming language, combined with a database like mongo or an ORM, allows you to create complex queries much more quickly compared to SQL. You can maybe go into stored procedures and start doing loops and recombining multiple queries in there, but programming languages like javascript etc are typically much nicer than those used in stored procedures. I guess this is where we differ. I've written many SQL q…
I agree that loops in SQL are not great. Loops in programming languages are fine obviously. There are just many more and nicer constructs in a programming language to manipulate data.
Maybe you have a root table that is anchoring your query, say a Staff table. The Staff table has a Manager column, which is another row in the Staff table. You then need to do a bunch of aggregate stuff. So in a programming language, you can maybe query the database 3 times, once for the staff you need, and then again for maybe shifts completed and so on.
You can then easily put the staff into a dictionary with virtually no code. Then you loop over the non-dictionary staff array, and you have something like:
for (var staff in shittyStaff) {
processedStaff.add({
manager: staffDic[staff.manager],
shiftsCompleted: shifts[staff.id].length,
shiftsWithManager: shifts[staff.id].filter(s => s.coworker == staff.manager.id).length
});
var staff = db.Staff
.Include(s => s.Manager)
.Include(s => s.Shifts)
.Select(s => new {
Manager: s.Manager,
ShiftsCompleted: s.Shifts.Count(),
ShiftsWithManager: s.Shifts.Where(shift => shift.CoWorker == s.Manager).Count()
});
I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…
Earlier quoted context omitted.
>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a b…
> Everybody uses an ORM with a SQL database. I'm ripping ORMs out of any backend code and using things like jOOQ. > I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more".... > Joins are slow and complicated You know what's really slow? Creating joins in code. > This is not true at all. Programming languages are vastly superior at querying a database. Just go write a comp…
It really isn't. In a good programming language expressing a join should be just as easy as in SQL.
> It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job.
Agreed that you need a way to do aggregate queries on the server, but plenty of NoSQL systems give you ways to do that. I'd far rather write a map-reduce in javascript/erlang/... than express the operation in SQL and hope that the query planner does what I think it's going to do (which ends up a bit like trying to program in Prolog - it's magic until the day it doesn't work and you then have no idea what's gone wrong or how to fix it).
Earlier quoted context omitted.
> it is super easy to program with javascript. It would be more honest to say that the weird js design, its lack of features and ridiculously small stdlib made it harder to do anything complicated at the time (no ES6 or webpack, remember ?). Hence, not good tooling existed for relational databases: no decent abstraction layer, migration lib or ORM. Lost of languages had a 2 liner to pop an sqlite db without needing t…
True that Node.JS didn't have, and still doesn't have a 2 liner to pop an sqlite db. But what it did have(a few years after its launch) was a pure-JS sqlite client which can leverage V8's event loop and libuv to make parallel execution of db-queries a trivial operation. There is a lot of mettle in Node.js for server side programming. PS: have you looked at TypeORM( http://typeorm.io/ )
However, it's not that relevant for most db workload. We had the same debate in Python: do we async our db libs or keep using threads ?
Experts say: http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an...
And this is despite the way the Python community adopted libuv with the uvloop lib.
TypeORM does look super nice, thanks. Does it handle well introspection ?
Earlier quoted context omitted.
Spanner is CP I think the A part comes from reliability of google network infrastructure which is not part of DB system's design per se.
I think I misunderstood that yes. https://storage.googleapis.com/pub-tools-public-publication-... . TLDR: It's technically CP, but google claims partitions are so rare people can assume it's CA as well (99.999% available). I assumed TrueTime bypassed the CAP theorem, but apparently that's marketing bs. It's to ensure something called external consistency, which is important if you want to take consistent snapshots ov…
Earlier quoted context omitted.
This is the problem with people that use the term NoSQL. Do you mean Cassandra (BigTable), MongoDB (Document), Riak (Key/Value), Redis (Mix), Kafka (Log Structured) ? There are dozens of fundamentally different systems many of which are closer to an RDBMS than their NoSQL peers. And many of them have rigid schemas so it definitely isn't that either.
You can also add Neo4j (graph) as part of the NoSQL family, right?
Graph databases arguably handle relational data better than SQL, and I'd argue they're much nicer to work with.
Earlier quoted context omitted.
Linq to SQL is still a thing, you may need to click it’s package when you install visual studio 2017+, but it’s still there and it’s still the best SQL interface they have build in my opinion. We rarely use anything else, but I can see the value of entity if you’re changing your DB a lot. We typically don’t do that though, and to be honest, working with changes with linq to sql, typically goes smoother than entity mi…
LINQ to SQL is a dead end. Look into LINQ to DB https://github.com/linq2db/linq2db which is way faster than LINQ to SQL ever was (faster than even Dapper). If you must you could give EF a try, but stay away from LINQ to SQL.
I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…
We now migrate to PG using 'core' attributes in tables and 'flexible' attributes in JSONB to get the best of two worlds.
I agree to the API problem, JDBC and PG/SQL is not as nice as a interace compared to Mongo libs.
If a system already has a well-defined domain, NoSQL adds little value.
Of course you could leverage AWS DynamoDB and reduce cost, but you still have downstream implications for things like reporting, which requires a known schema with relational paradigms.