Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RediSQL – A Redis module that provides a functional SQL database

github.com

Post reply on HN
Print view View original

31–40 of 60 posts

Re: RediSQL – A Redis module that provides a functional SQL database

#31

Post by m82labs »

nolok wrote: post #28
m82labs wrote: post #27

Fully in-memory SQL can also be accomplished using IMOLTP via MS SQL Express on Linux now as well. And you get a fairly feature complete engine along with multithreaded execution, no blocking, and some great client options. The only limits right now are 4 cores and ~350MB of in-memory data per database.

My memory is a bit fuzzy right now, but isn't mysql itself providing that since forever too? Or do you fully in memory db - with a hard copy on the hard drive?

I think most RDBM systems have a way to pin a table or database in memory. The difference with the Microsoft solution is that it is a separate engine designed explicitly for memory. So there is no locking or blocking of pages. There are some other neat features as well, like hash indexes, but I have been able to push an 8 core box to ~80,000 inserts per second from a remote client, and over 200,000 per second for a test I ran on the host. I have not yet ran tests for express edition, but I will be interested to see how it performs.

To answer your other question, you have the option to either keep it all in memory (and lose on reboot) or commit the data to disk. In my testing I only saw a ~10% performance penalty for committed data to disk (granted we have extremely fast SSDs).

Re: RediSQL – A Redis module that provides a functional SQL database

#32

Post by EugeneOZ »

EugeneOZ wrote: post #30

I tried to do the same (before great Lua support, using PHP), but overall performance was similar to MySQL (much more reliable than my experiment) so I abandoned it. So would be very interesting to see benchmarks and comparisons with MySQL - I think right now we have more ways to implement it better.

Really interesting to hear reasons of downvoting. You think I'm lying or there is comparison?

Re: RediSQL – A Redis module that provides a functional SQL database

#33

Post by nik736 »

EugeneOZ wrote: post #32
EugeneOZ wrote: post #30

I tried to do the same (before great Lua support, using PHP), but overall performance was similar to MySQL (much more reliable than my experiment) so I abandoned it. So would be very interesting to see benchmarks and comparisons with MySQL - I think right now we have more ways to implement it better.

Really interesting to hear reasons of downvoting. You think I'm lying or there is comparison?

PHP.

Re: RediSQL – A Redis module that provides a functional SQL database

#34

Post by dvirsky »

I've been working on a secondary indexing module for redis with a subset of SQL WHERE for selection predicates, if anyone's interested. It's not stable yet, but it's coming along nicely.

The idea was actually NOT to create an SQL like database, but just proxy ordinary redis commands via secondary index. So you select ids, and perform HGET or HGETALL on them, etc. And the same goes for indexing - you perform something like HMSET and "through" the index, and it performs the operation while indexing the hash automatically.

Also, indexes can be used as direct data types with no automation, you just insert ids and tuples of data into them, and you can query them and get ids back. https://github.com/RedisLabsModules/secondary

Re: RediSQL – A Redis module that provides a functional SQL database

#35

Post by camtarn »

alfanick wrote: post #18
aalekhN wrote: post #15

Earlier quoted context omitted.

Have you thought of ways to implement queries involving LIKE operations ? I worked on a similar project some time ago and one of the limitation for such implementation I encountered was use of glob-style patterns in redis.

This is just a bridge to in-memory sqlite accessible from redis connection – it does not emulate sql for redis data - both databases are completely independent, unfortunately.

This paragraph is what I needed to see in the project readme. Thanks.

Re: RediSQL – A Redis module that provides a functional SQL database

#36

Post by [deleted] »

alfanick wrote: post #18
aalekhN wrote: post #15

Earlier quoted context omitted.

Have you thought of ways to implement queries involving LIKE operations ? I worked on a similar project some time ago and one of the limitation for such implementation I encountered was use of glob-style patterns in redis.

This is just a bridge to in-memory sqlite accessible from redis connection – it does not emulate sql for redis data - both databases are completely independent, unfortunately.

[deleted]

Re: RediSQL – A Redis module that provides a functional SQL database

#37

Post by EugeneOZ »

nik736 wrote: post #33
EugeneOZ wrote: post #32

Earlier quoted context omitted.

Really interesting to hear reasons of downvoting. You think I'm lying or there is comparison?

PHP.

PHP wasn't a bottleneck in this case, most of the time has been spent on communication with DB (as usually in web apps). I think (and hope) this solution can show better performance because of less communications required, but numbers are more interesting than assumptions.

upd: comment about proxying sqlite eliminated this hope.

Re: RediSQL – A Redis module that provides a functional SQL database

#38

Post by antirez »

I think that what the module does is actually not clear here in the comments I'm reading. It's mostly a nice "Hello World" module that shows how Redis can act as a proxy, but it's just a small glue-module that calls the SQLite library with the string passed as query. So you are running queries on a local sqlite server, proxied by Redis.

While this can be a good programming example for newcomers, I suggest to look at modules that are really implementing new DB paradigms on top of Redis. For instance this module builds a graph DB on top of Redis data types, with a query language and so forth:

https://github.com/swilly22/redis-module-graph

Re: RediSQL – A Redis module that provides a functional SQL database

#39

Post by arielm »

antirez wrote: post #38

I think that what the module does is actually not clear here in the comments I'm reading. It's mostly a nice "Hello World" module that shows how Redis can act as a proxy, but it's just a small glue-module that calls the SQLite library with the string passed as query. So you are running queries on a local sqlite server, proxied by Redis. While this can be a good programming example for newcomers, I suggest to look at…

I completely agree. As someone who uses redis sql heavily I understand both have their use cases and wouldn't want to merge both to begin with.

On top of that, this looks like a simple proxy, so it's only making things less efficient.

OP, I'd suggest explaining the benefits better. Maybe I missed it but I don't really get this at all.

Re: RediSQL – A Redis module that provides a functional SQL database

#40

Post by siscia »

arielm wrote: post #39
antirez wrote: post #38

I think that what the module does is actually not clear here in the comments I'm reading. It's mostly a nice "Hello World" module that shows how Redis can act as a proxy, but it's just a small glue-module that calls the SQLite library with the string passed as query. So you are running queries on a local sqlite server, proxied by Redis. While this can be a good programming example for newcomers, I suggest to look at…

I completely agree. As someone who uses redis sql heavily I understand both have their use cases and wouldn't want to merge both to begin with. On top of that, this looks like a simple proxy, so it's only making things less efficient. OP, I'd suggest explaining the benefits better. Maybe I missed it but I don't really get this at all.

The modules is born just to fulfill one of our needs: we had several micro-services using redis and no DB at all, but then we find ourselves in the situation where we needed SQL capabilities, redis module where being announced and we give them a shoot.

What you mean by efficiency ? Operation at the second? Sure it is less efficient than Postgres or of SQLite embed in your application.

However if you mean convenience and simplicity this solution may (arguably) be efficient enough.

This module sit between a "full" client-server DB, a redis instance (that doesn't provide SQL capabilities) and SQLite that is not client-server.

Being completely honest, how HN teach us, build a great technical product without showing it as soon as it provide just a tiny amount of value is not desirable.

The comment in this thread indicate that there is some interest and I got some valuable suggestions and feedback on what to develop next.

Post reply on HN