Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RediSQL – A Redis module that provides a functional SQL database

github.com

Post reply on HN
Print view View original

51–60 of 60 posts

Re: RediSQL – A Redis module that provides a functional SQL database

#51

Post by Jweb_Guru »

badminton1 wrote: post #50

Redis and NoSQL data stores in general are for merge-on-write not merge-on-read. That's why most of them are non relational (no relations, foreign keys or joins). This project encourages merge-on-read which is exactly the problem k/v stores try to solve in another way (merge-on-write). The problem with relational data stores is that once you have relations then you need consistency and then you need moving operations…

The problem of consistency is related to supporting multi-key, read-write transactions of any sort, and has nothing to do with support for joins, or relations. You're not escaping from it somehow by using a NoSQL database.

Re: RediSQL – A Redis module that provides a functional SQL database

#52

Post by badminton1 »

Jweb_Guru wrote: post #51
badminton1 wrote: post #50

Redis and NoSQL data stores in general are for merge-on-write not merge-on-read. That's why most of them are non relational (no relations, foreign keys or joins). This project encourages merge-on-read which is exactly the problem k/v stores try to solve in another way (merge-on-write). The problem with relational data stores is that once you have relations then you need consistency and then you need moving operations…

The problem of consistency is related to supporting multi-key, read-write transactions of any sort, and has nothing to do with support for joins, or relations. You're not escaping from it somehow by using a NoSQL database.

In a relational database you are almost certainly going to maintain data about the same "entity" distributed across different tables. And that is encouraged (hence the term "relational databases").

In k/v datastores, e.g: document oriented, you can just put all the data related to one entity on one record and that's it... and dealing with multi-key operations is discouraged rather than encouraged.

Why discouraged? because it's hard to load balance, creates the need for synchronization and locking at a broader level, adds complexity to the maintenance operations, and so on, so forth.

The SQL thing exists because dBase, one of the first database products, had it. And then every product that entered the market had to go with it to diminish friction.

Re: RediSQL – A Redis module that provides a functional SQL database

#53

Post by edibleEnergy »

pizza234 wrote: post #24
siscia wrote: post #3

Author here, if you have any question please feel free to ask. Also I have introduced the module in a blog post here: http://redbeardlab.tech/2016/12/13/redisql

Which would be a use case, from a technical perspective? The blog mentions: If you are writing microservices to avoid bothering the master of DB with your need ephemeral needs of structured database and I see this module perfect for small dataset of unimportant data. In other words, why would one use this module in place of a straight SQLite database (possibly in-memory, if there is really such need for speed)? Such…

Well, SQLite has to be accessed over a filesystem. This sounds like it would proxy to a sqlite db located on the Redis server. Still sounds dicey but I could see the appeal if you didn't have the DBA skills necessary to set up a decent PostgreSQL or other SQL server.

Re: RediSQL – A Redis module that provides a functional SQL database

#54

Post by [deleted] »

EugeneOZ wrote: post #37
nik736 wrote: post #33

Earlier quoted context omitted.

PHP.

PHP wasn't a bottleneck in this case, most of the time has been spent on communication with DB (as usually in web apps). I think (and hope) this solution can show better performance because of less communications required, but numbers are more interesting than assumptions. upd: comment about proxying sqlite eliminated this hope.

[deleted]

Re: RediSQL – A Redis module that provides a functional SQL database

#55

Post by Jweb_Guru »

badminton1 wrote: post #52
Jweb_Guru wrote: post #51

Earlier quoted context omitted.

The problem of consistency is related to supporting multi-key, read-write transactions of any sort, and has nothing to do with support for joins, or relations. You're not escaping from it somehow by using a NoSQL database.

In a relational database you are almost certainly going to maintain data about the same "entity" distributed across different tables. And that is encouraged (hence the term "relational databases"). In k/v datastores, e.g: document oriented, you can just put all the data related to one entity on one record and that's it... and dealing with multi-key operations is discouraged rather than encouraged. Why discouraged? be…

Okay, so your solution to consistency is "grab a big ol' lock over all possible related data, and hope like hell that you got all of it and it can live in one key." That doesn't really work when the amount of related data you have gets large enough, and it's really easy to have an efficient SQL database behind the lock (that's essentially how VoltDB works). Behind the lock, you can still have multiple tables, joins, and everything people associate with RDBMSes, without any of the consistency issues you mention.

Your discussion about what's "encouraged" or not is frankly irrelevant, as key/value stores are super easy compared to general relational databases and most RDBMSes are already quite good key/value stores (by contrast, most key/value stores make lousy RDBMSes). The simple fact of the matter is: if your key/value store doesn't support multikey read/write transactions, it has strictly less functionality than a database that does, regardless of how you use it, and if it does support multikey read/write transactions it'll have all the same performance issues you associate with RDBMSes (well, that's a slight oversimplification because it may provide a more limited API).

Re: RediSQL – A Redis module that provides a functional SQL database

#56

Post by pizza234 »

edibleEnergy wrote: post #53
pizza234 wrote: post #24

Earlier quoted context omitted.

Which would be a use case, from a technical perspective? The blog mentions: If you are writing microservices to avoid bothering the master of DB with your need ephemeral needs of structured database and I see this module perfect for small dataset of unimportant data. In other words, why would one use this module in place of a straight SQLite database (possibly in-memory, if there is really such need for speed)? Such…

Well, SQLite has to be accessed over a filesystem. This sounds like it would proxy to a sqlite db located on the Redis server. Still sounds dicey but I could see the appeal if you didn't have the DBA skills necessary to set up a decent PostgreSQL or other SQL server.

Well, although the requirements which would lead to such setup are clearer now, it's still smelly to me.

Under this conditions there is still a false dichotomy between RediSQL on one side, and "complex RDBMS" on the other.

What about simple RDMSs like H2? Such solution would meet the requirements:

    - network-accessible RDBMS
    - no configuration
The only requirement I can think of which would make this solution inadequate would be "No JVM on the system", which sound quite far-fetched.

Re: RediSQL – A Redis module that provides a functional SQL database

#57

Post by dvirsky »

badminton1 wrote: post #50

Redis and NoSQL data stores in general are for merge-on-write not merge-on-read. That's why most of them are non relational (no relations, foreign keys or joins). This project encourages merge-on-read which is exactly the problem k/v stores try to solve in another way (merge-on-write). The problem with relational data stores is that once you have relations then you need consistency and then you need moving operations…

It's not even that in this case - it's just SQLite with a network layer, no persistence and each operation blocks redis completely until finished.

Re: RediSQL – A Redis module that provides a functional SQL database

#58

Post by arielm »

siscia wrote: post #49
antirez wrote: post #47

Earlier quoted context omitted.

Hey, glad to know you are going multi-threaded on that, since I believe it will be very easy. To start you could just use a thread per request (not just to start, probably even in the long run), immediately block the client via the module API to do this, fire the thread, wait for the thread to complete the work and accumulate the reply, and re-surrect the blocked client, feeding the reply. AFAIK this can work very we…

I tried to spawn a new thread for each request and it was embarrassedly slow, maybe I was doing something wrong from my side, but I believe I will settle for a pool. About the reply the only thing that I see manageable is to include the name of the columns as first row, I was considering the idea but I am not completely sold yet. I was thinking to bind a particular statement to a key in order to provide the possibili…

Just to throw my two cents into the mix. The idea of being able to _query_ a single key in a SQL-like fashion is interesting, but I can't think of a production implementation that would be useful enough.

Instead, I'd look at using redis in front of something like MySQL so some operations go to redis and other right to the server.

Having to write SQL statements as redis EXECs isn't elegant IMO.

Re: RediSQL – A Redis module that provides a functional SQL database

#59

Post by [deleted] »

siscia wrote: post #49
antirez wrote: post #47

Earlier quoted context omitted.

Hey, glad to know you are going multi-threaded on that, since I believe it will be very easy. To start you could just use a thread per request (not just to start, probably even in the long run), immediately block the client via the module API to do this, fire the thread, wait for the thread to complete the work and accumulate the reply, and re-surrect the blocked client, feeding the reply. AFAIK this can work very we…

I tried to spawn a new thread for each request and it was embarrassedly slow, maybe I was doing something wrong from my side, but I believe I will settle for a pool. About the reply the only thing that I see manageable is to include the name of the columns as first row, I was considering the idea but I am not completely sold yet. I was thinking to bind a particular statement to a key in order to provide the possibili…

[deleted]

Re: RediSQL – A Redis module that provides a functional SQL database

#60

Post by siscia »

arielm wrote: post #58
siscia wrote: post #49

Earlier quoted context omitted.

I tried to spawn a new thread for each request and it was embarrassedly slow, maybe I was doing something wrong from my side, but I believe I will settle for a pool. About the reply the only thing that I see manageable is to include the name of the columns as first row, I was considering the idea but I am not completely sold yet. I was thinking to bind a particular statement to a key in order to provide the possibili…

Just to throw my two cents into the mix. The idea of being able to _query_ a single key in a SQL-like fashion is interesting, but I can't think of a production implementation that would be useful enough. Instead, I'd look at using redis in front of something like MySQL so some operations go to redis and other right to the server. Having to write SQL statements as redis EXECs isn't elegant IMO.

> Just to throw my two cents into the mix.

You have no idea how much those 2 cents are appreciated, really, I mean it.

> The idea of being able to _query_ a single key in a SQL-like fashion is interesting

Not sure I understood what you mean here. Would you mind to provide some kind of example or to explain it further?

> Instead, I'd look at using redis in front of something like MySQL so some operations go to redis and other right to the server.

Extremely interesting, how you discriminate though ? If it is going to be a "regular" cache you don't need a module, redis can already do it by itself. What use case you were think of?

> Having to write SQL statements as redis EXECs isn't elegant IMO.

I do completely agree, the API is going to get more structured for sure.

Post reply on HN