Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Can Do Better Than SQL

edgedb.com

Post reply on HN
Print view View original

451–460 of 466 posts

Re: We Can Do Better Than SQL

#451

Post by SahAssar »

lmm wrote: post #435
marcosdumay wrote: post #346

Earlier quoted context omitted.

As long as your application layer is a single homogeneous application running the same code. That is approximately never the case. Current widespread practices are moving away from it, with multi-services, and old fashioned practices of mixing customized and off the shelf tools basically forbid it.

Current practice is moving away from the idea of having your storage layer be a single homogeneous datastore as well.

So where are you proposing the consensus be placed? A lot (I'd argue most) applications need consensus for them to be stable and reliable in the long term, so either you place it in the datastore or in the logic processing. Where else would you place it?

Re: We Can Do Better Than SQL

#452

Post by SahAssar »

yen223 wrote: post #117
CaptainZapp wrote: post #95

Earlier quoted context omitted.

> I will cheer everyone who tries to displace SQL Why? What are better alternatives, really?

There's a non-trivial number of live running systems out there that interact with RDBMSes entirely through either ORMs or query builders, without a single line of programmer-written SQL. Those systems seem to hold up well.

Those don't replace SQL in the same way that typescript does not replace javascript. They are also often specific to a certain framework or language in a way that makes them infeasible to compete with SQL in a general way.

Re: We Can Do Better Than SQL

#453

Post by SahAssar »

seer wrote: post #129

I for one would welcome a new alternative to sql. It might not be _this_ alternative, but why not try. SQL is very hard to learn properly, with all of its gotchas and inconsistencies. There are running jokes for noobs truncating their tables due to forgetting a where clause. I’ve seen junior devs crying in tears and throwing their mice just because they needed to debug / optimise a complex query. The mare existence o…

> There are running jokes for noobs truncating their tables due to forgetting a where clause. I’ve seen junior devs crying in tears and throwing their mice just because they needed to debug / optimise a complex query.

Isn't this the same for most shells including almost all linux distros? The CEO of red hat accidentally wiped his computer becuase he forgot a slash but we don't throw out a whole tool just because it included a footgun.

The same is true for SQL.

Re: We Can Do Better Than SQL

#455

Post by lmm »

SahAssar wrote: post #451
lmm wrote: post #435

Earlier quoted context omitted.

Current practice is moving away from the idea of having your storage layer be a single homogeneous datastore as well.

So where are you proposing the consensus be placed? A lot (I'd argue most) applications need consensus for them to be stable and reliable in the long term, so either you place it in the datastore or in the logic processing. Where else would you place it?

I prefer to see the whole system as a succession of stream transformations (https://www.confluent.io/blog/turning-the-database-inside-ou...). If you view the sequence of input events as first-class and the "current state of the world" as derived, then a lot of problems go away. You need a datastore that can give you a consistent answer as to what order events occurred in, but it's a lot easier to make appending to an append-only list atomic than to make arbitrary state computations ACID.

For downstream derived computations, basically you either make the causal relationship explicit, or accept that you have eventual consistency. The only case where you can have inconsistency is where you have a "diamond" in your computations (i.e. you compute B that's derived from A and C that's also derived from A, and you compute D that's derived from B and C). So you figure out the business implications and either accept it or eliminate the diamond (by computing (A, B) from A and (B,C) from (A, B) instead of computing B and C separately from A). You will also get inconsistency if you do some totally ad-hoc query that's not part of your existing pipelines, but usually that's the kind of reporting query that doesn't need to be 100% consistent; if you do need a consistent version of that then the best approach is to take a regular snapshot, which can be consistent.

Basically you have a lot more precise control, you have causal relationships where you have explicit dependencies, so you have the level of consistency that you need for all your operational stuff, but you don't have a globally consistent realtime view of everything. It may take more work up front, but IME the notion of that kind of global consistency is a lie in a distributed world; even if you just have a basic webapp then you can't actually achieve the kind of consistency that that model pretends you have, because what the user is viewing in their browser (and then potentially making changes based on) at any given time is not necessarily the same as what's in the central database.

To get back to the question, this means that if inconsistency is due to an actual bug, it's pretty easy to solve: fix the bug and then regenerate from the original events. If you don't understand why data is inconsistent then you can always look back to the source events and figure out what it should be.

Re: We Can Do Better Than SQL

#456

Post by okaleniuk »

throwaway_pdp09 wrote: post #450
okaleniuk wrote: post #443

Earlier quoted context omitted.

For instance, we choose Riak+LevelDB some 8 years ago as our data store. It doesn't support SQL but it's fast and easily scalable. And free. I'm not sure if we could achieve the same speed and fault tolerance with any SQL solution available at that point. So the deal is, you give up your time invested in learning SQL and get some advantages in return. This is reasonable. But if it's only about language, no additional…

With respect, I think you're missing a lot. Sit with an expert SQL guy and you'll learn a great deal. Maybe not enough to change your mind but certainly enough to realise you're blocking out a really valuable technology. > but languages are just thin interfaces over technologies No. Not at all. SQL is derived from a mathematical basis. The principle came first, then the language and tech together developed to fulfil…

But that doesn't contradict my point. The mathematical basis matters, sure, but the language on top of it (exactly because it derives from the math anyway) is mostly irrelevant.

I don't really care if I have to write SELECT or \forall or σ as long as they all have the same meaning. But I'd prefer to use SELECT because I already spent time learning how to work this way.

SQL as a language is fine. There is no real need to reinvent SQL unless you're going to reinvent the math behind it too.

Re: We Can Do Better Than SQL

#457

Post by lostjohnny »

013a wrote: post #360
lostjohnny wrote: post #353

Earlier quoted context omitted.

> I would point to MongoDB's query language seriuosly? db.orders.aggregate([ { $lookup: { from: "warehouses", let: { order_item: "$item", order_qty: "$ordered" }, pipeline: [ { $match: { $expr: { $and: [ { $eq: [ "$stock_item", "$$order_item" ] }, { $gte: [ "$instock", "$$order_qty" ] } ] } } }, { $project: { stock_item: 0, _id: 0 } } ], as: "stockdata" } } ]) VS SELECT *, stockdata FROM orders WHERE stockdata IN (SE…

If you try to directly translate SQL to MongoDB, without changing your schemas or query, then yes, it's gonna look bad. That doesn't make for a good comparison, and I think you know that.

> That doesn't make for a good comparison, and I think you know that.

That query is not my invention, it comes directly from MongoDB documentation

If it looks bad, it means it is bad by design

https://docs.mongodb.com/manual/reference/operator/aggregati...

Re: We Can Do Better Than SQL

#458

Post by lostjohnny »

013a wrote: post #406
petepete wrote: post #359

Earlier quoted context omitted.

Another huge win for SQL is that it's easy to construct from parts. You can very easily run and debug your subquery or common table expression on its own before combining it into a larger, more-complex query. If you (as I usually do) create plenty of views while analysing a dataset, the approach can be extremely powerful. Doing the same in JavaScript is possible, but it's slow and cumbersome by comparison.

The original article addresses the deficiency in construction from parts, in its "Lack of Orthogonality" section. MongoDB queries, while being interpretable by javascript, aren't really javascript. You can't interact with the data using javascript (well, you can, using eval, but you shouldn't). You interact with the data via the query language, which is, again, expressed in JS, just like SQL is expressed in English.…

> with no string manipulation or ORM

OR, another way to look at it, using MongoDB's ORM, which is quite bad if you ask me.

Re: We Can Do Better Than SQL

#459

Post by wtetzner »

achr2 wrote: post #398
wtetzner wrote: post #390

Earlier quoted context omitted.

That doesn't make the relation model bad, it just means there's an impedance mismatch. FWIW, I find the relational model to be much nicer than typical object models. Even in Java, which I write every day at work, I've found it to be much clearer to use immutable objects representing records. Not just for interacting with the database, but just for handling information in general. Of course, it would be extremely pain…

I agree that the mismatch is not a direct failing of SQL or the relational model - but we continue to use it without adopting a better approach or developing a common abstraction layer. With all of the syntax growth, SQL could certainly have a join syntax that understands object composition and returns structured data.

> With all of the syntax growth, SQL could certainly have a join syntax that understands object composition and returns structured data.

I mean, many RDBMS's have support for JSON and/or XML, so you can sort of get it.

However, I wonder if we'd be better of as an industry if we dealt with our data in code in a more relation-y way than an object-y way.

Re: We Can Do Better Than SQL

#460

Post by throwaway_pdp09 »

okaleniuk wrote: post #456
throwaway_pdp09 wrote: post #450

Earlier quoted context omitted.

With respect, I think you're missing a lot. Sit with an expert SQL guy and you'll learn a great deal. Maybe not enough to change your mind but certainly enough to realise you're blocking out a really valuable technology. > but languages are just thin interfaces over technologies No. Not at all. SQL is derived from a mathematical basis. The principle came first, then the language and tech together developed to fulfil…

But that doesn't contradict my point. The mathematical basis matters, sure, but the language on top of it (exactly because it derives from the math anyway) is mostly irrelevant. I don't really care if I have to write SELECT or \forall or σ as long as they all have the same meaning. But I'd prefer to use SELECT because I already spent time learning how to work this way. SQL as a language is fine. There is no real need…

2 points then;

1. Do you know SQL currently? Because your previous comments implied you don't

2. SQL is a bad implementation of the underlying relational model. And it's just bad in other ways. For one, it's bloody wordy:

  select * from people
but as SQL is an expression language, why not?

  people
From an answer of mine a few days ago (slightly modified here) to pick out differences:

  (
  select *
  from t1
  except
  select *  from t2
  )
  union 
  (
  select *
  from t2
  except
  select *
  from t1
  )
I'd prefer to write that crap as

(t1 \ t2) + (t2 \ t1)

or similar. If you write a lot of SQL, that matters.

SQL aggregates don't nest easily. Transactions are screwed up (https://news.ycombinator.com/item?id=23569513). SQL is poorly composable (https://news.ycombinator.com/item?id=23550420#23561301). For better or worse it has nulls therefore tristate logic. This causes errors! And more.

BTW. transactions don't work in the way you might think, atomically. Which is why MS added https://docs.microsoft.com/en-us/sql/t-sql/statements/set-xa... and I'm sure Pgres has similar. From the link:

"When SET XACT_ABORT is OFF, in some cases only the Transact-SQL statement that raised the error is rolled back and the transaction continues processing"

That peculiarity is from the standard BTW.

SQL is problematic. It could have been much better.

Post reply on HN