Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis: the AK-47 of databases

flazz.me

Post reply on HN
Print view View original

21–30 of 82 posts

Re: Redis: the AK-47 of databases

#21

Post by liuliu »

IgorPartola wrote: post #12
code_duck wrote: post #3

Earlier quoted context omitted.

According to the first comment, the part about locking the whole data set for MULTI is not accurate. Only the connection calling that is stalled, which makes perfect sense.

Ah: http://redis.io/commands/multi . So the problem with this approach is that there are no consistent reads. If I say: MULTI get foo # Returns 'abc' set foo 'abcd' # (as in application code said foo += 'd') EXEC while another process sets foo to 'xyz' between my get and set, I just overwrote that value. No serializable transactions.

Redis is single-thread one-process program. Since the execution only happens when EXEC command received, such scenario cannot exist.

Re: Redis: the AK-47 of databases

#22

Post by wingo »

adamtj wrote: post #20

"The entire dataset is in RAM." How can they guarantee that, when RAM is little more than a disk cache on modern operating systems? Is there a syscall that prevents specified pages of virtual memory from being paged out to disk? Or is Redis implemented as a kernel module?

Your swap should be off, and there is mlockall(2).

The kernel does let userspace have a /little/ fun on its own.

Re: Redis: the AK-47 of databases

#23

Post by antirez »

adamtj wrote: post #20

"The entire dataset is in RAM." How can they guarantee that, when RAM is little more than a disk cache on modern operating systems? Is there a syscall that prevents specified pages of virtual memory from being paged out to disk? Or is Redis implemented as a kernel module?

Redis uses a memory layout that, unless the instance is completely idle, will make it very easy to touch every page even if there are just a few queries per second.

So fare in two years we never saw an instance where there was a latency problem because part of the dataset was swapped on disk by the OS. This is why we currently don't use mlockall().

Re: Redis: the AK-47 of databases

#24

Post by antirez »

IgorPartola wrote: post #12
code_duck wrote: post #3

Earlier quoted context omitted.

According to the first comment, the part about locking the whole data set for MULTI is not accurate. Only the connection calling that is stalled, which makes perfect sense.

Ah: http://redis.io/commands/multi . So the problem with this approach is that there are no consistent reads. If I say: MULTI get foo # Returns 'abc' set foo 'abcd' # (as in application code said foo += 'd') EXEC while another process sets foo to 'xyz' between my get and set, I just overwrote that value. No serializable transactions.

MULTI/EXEC is completely atomic and transactional.

You can perform just writes inside the block. If you want atomicity with reads in the middle, you need to use WATCH/MULTI/EXEC that uses an optimistic locking algorithm.

Re: Redis: the AK-47 of databases

#25

Post by keiferski »

ErrantX wrote: post #10

The title randomly reminded me of the AK-47 quote from Lord of War (an actually decent Cage film, and definitely a good bit of monologue): Of all the weapons in the vast soviet arsenal, nothing was more profitable than Avtomat Kalashnikova model of 1947. More commonly known as the AK-47, or Kalashnikov. It's the world's most popular assault rifle. A weapon all fighters love. An elegantly simple 9 pound amalgamation o…

Supposedly the Ak 47s used in the film were real, as real Ak 47s are less expensive than fake ones. But I'm not sure where I heard this, or if it's even true.

Re: Redis: the AK-47 of databases

#26

Post by simonw »

jamesli wrote: post #16

The author needs to know more about databases before claiming something is the AK-47 of the subject. It kinds of reminds me of those guys who think they are gurus just because they know some rarely used Linux commands, while in fact they don't have good understanding of OS. :)

What do you think he's missing? The comparison seems rather apt to me - Redis is shockingly simple and surprisingly effective compared to other persistence options.

Re: Redis: the AK-47 of databases

#27

Post by Luc »

keiferski wrote: post #25
ErrantX wrote: post #10

The title randomly reminded me of the AK-47 quote from Lord of War (an actually decent Cage film, and definitely a good bit of monologue): Of all the weapons in the vast soviet arsenal, nothing was more profitable than Avtomat Kalashnikova model of 1947. More commonly known as the AK-47, or Kalashnikov. It's the world's most popular assault rifle. A weapon all fighters love. An elegantly simple 9 pound amalgamation o…

Supposedly the Ak 47s used in the film were real, as real Ak 47s are less expensive than fake ones. But I'm not sure where I heard this, or if it's even true.

Well, have I got the definite resource for you! http://www.imfdb.org/index.php/Lord_of_War

"According to Director Andrew Niccol in the DVD commentary, the guns were real guns rented from a real arms dealer, as it was cheaper for the production to rent 3,000 real guns than to rent 3,000 blank converted props. "

Re: Redis: the AK-47 of databases

#28

Post by Duff »

ErrantX wrote: post #10

The title randomly reminded me of the AK-47 quote from Lord of War (an actually decent Cage film, and definitely a good bit of monologue): Of all the weapons in the vast soviet arsenal, nothing was more profitable than Avtomat Kalashnikova model of 1947. More commonly known as the AK-47, or Kalashnikov. It's the world's most popular assault rifle. A weapon all fighters love. An elegantly simple 9 pound amalgamation o…

I doubt the actual weapon was profitable. They literally made like 75 million of the things. The Soviets may have made a few bucks setting up factories in the 3rd world for licensed production.

Re: Redis: the AK-47 of databases

#29

Post by IgorPartola »

antirez wrote: post #24
IgorPartola wrote: post #12

Earlier quoted context omitted.

Ah: http://redis.io/commands/multi . So the problem with this approach is that there are no consistent reads. If I say: MULTI get foo # Returns 'abc' set foo 'abcd' # (as in application code said foo += 'd') EXEC while another process sets foo to 'xyz' between my get and set, I just overwrote that value. No serializable transactions.

MULTI/EXEC is completely atomic and transactional. You can perform just writes inside the block. If you want atomicity with reads in the middle, you need to use WATCH/MULTI/EXEC that uses an optimistic locking algorithm.

Ah, that makes sense. I imagine allowing atomic reads without WATCH would complicate the code quite a bit.

Re: Redis: the AK-47 of databases

#30

Post by IgorPartola »

MichaelStubbs wrote: post #18
IgorPartola wrote: post #2

Eek. Locking the entire data set to perform transactions? Isn't that so... MyISAM? Does rolling the transaction back work? The optimistic lock is not a great solution either: instead of deadlocking while waiting for a lock to be released, you are now stuck in a while (!saved) loop. Imagine two web pages trying to update the same resource at the same time. Also this part of the config: save 900 1 save 300 10 save 60 1…

The configuration options you have quoted are made extremely obvious in the sample configuration and are explained very well. See: # Save the DB on disk: # # save # # Will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # In the example below the behaviour will be to save: # after 900 sec (15 min) if at least 1 key changed # after 300 sec (5 min) i…

My point is the OP talks about how all the options are self-evident.
Post reply on HN