Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis 4.0

groups.google.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Redis 4.0

#131

Post by ericfrederich »

antirez wrote: post #7
Dowwie wrote: post #4

@antirez: Congrats! Are you going to modularize disque now that v4 is ready?

Yes, one of the top items in the Redis 4.2 roadmap is to port Disque as a Redis module, so the steps are: 1) Implement a Redis Cluster modules API (I've already some draft design, basically there are low level stuff to enlist nodes, broadcast messages, send one-to-one messages, and have callbacks called when we receive a message), and then port Disque as Redis module on top of such API, so that we can both validate t…

I look forward to Aphyr scrutinizing it. No disrespect to you, his posts are always fun to read and show how hard it is to get this stuff correct.

Re: Redis 4.0

#132

Post by ihsw2 »

jackmott wrote: post #122
xxs wrote: post #84

Earlier quoted context omitted.

120GB on a single thread, yeah. Redis has been abused for quite some time but 120GB is way out of reasonable reach for current CPU arch if you use a single core only (even if you switch off hyper threading)

just randomly curious here - why? hash time gets absurd?

Presumably, time complexity for most operations correspond directly with CPU usage.

See here for an example with RPUSH: https://redis.io/commands/rpush

RPUSH is O(1) which the best you can possibly get.

ZADD on the other hand is O(log(n)), which is quite good but at a large scale it becomes easier to run into performance limitations, especially on high workloads. Performing O(n) operations on large data sets is out of the question unless you're comfortable with Redis being unavailable for multiple seconds/minutes.

That said, you will have to limit yourself to operations that are computationally simple (ie: O(1) or O(log(log(n))) or pre-shard across multiple instances (or run a Redis cluster).

Re: Redis 4.0

#133

Post by ing33k »

infocollector wrote: post #79

Is there a Redis PPA for Ubuntu 16.04 that is supported by the redis team?

Redis is of the the very few applications , that I install manually almost every time.

trick is to use the utils/install_server.sh

Re: Redis 4.0

#134

Post by ihsw2 »

FLUX-YOU wrote: post #112
ihsw2 wrote: post #17

Earlier quoted context omitted.

Memory fragmentation is largely related to the allocator you're using (ie: glibc malloc, jemalloc, tcmalloc) and previously it was up to the OS to manage this (ie: freeing up unused memory). Now with active memory defragmentation things are a bit more pleasant, specifically with high delete load actually freeing up unused memory in a timely manner without impacting performance too much. Previously, to fully recover u…

>Now with active memory defragmentation things are a bit more pleasant, specifically with high delete load actually freeing up unused memory in a timely manner without impacting performance too much. Is Redis managing this itself using simple calls to Jemalloc, or is Jemalloc doing it on its own because it has better algorithms than the OS?

With regards to better algorithms than the OS -- jemalloc is not quite there yet.

Here is an illuminating discussion between Redis and Jemalloc devs regarding this:

https://github.com/jemalloc/jemalloc/issues/566

Redis will perform its own housekeeping, hence the usage of the term 'active.' AFAIK, rather than metadata being stored for this, there is a periodic active scan and manual measurement.

See more here:

https://github.com/antirez/redis/pull/3720

Re: Redis 4.0

#135

Post by bluGill »

xxs wrote: post #90
jackmott wrote: post #59

Earlier quoted context omitted.

It is one of those things where people say "Java can be faster than C/C++ because it can compact memory" and the C/C++ lady says "here, hold my beer"

To be fair only a compacting GC can do that

That isn't strictly true. You do not need to do garbage collection to compact memory. The data needed to do memory compaction and garbage collection largely overlaps, so you may as well do both, but you don't have to. To implement either you need to do a scan of all pointers in memory - for a garbage collection you save a currently in use list, for compaction you need to a map of where they are so you can change them latter. What you do with that data is different, but only the first step is the same. (though garbage collection can feed the empty places into the compactor which makes that work easier)

I've actually been thinking about how to do compaction in C++. The reason you cannot do it in general is code like "socket.send(new object, sizeof(void*)". The remote is then expected to send back that pointer which you cast to object and reference. Evil code for sure with a lot of failure cases that result in leaks or security holes, but possible.

Re: Redis 4.0

#136

Post by [deleted] »

lozenge wrote: post #130
iagooar wrote: post #120

Earlier quoted context omitted.

Call me a slowpoke, but I wasn't aware that Redis was single core!

It makes it great for storing distributed locks, semaphores, and thanks to Lua scripting it can store/update cache invalidation lists - eg https://github.com/Suor/django-cacheops

[deleted]

Re: Redis 4.0

#137

Post by Others »

zedpm wrote: post #127
juliensab wrote: post #103

Earlier quoted context omitted.

Why?

One less service to manage, plus RabbitMQ's options for dealing with network partitions are all terrible. If you want to run a RabbitMQ cluster, you don't have any option that avoids data loss.

Can you even avoid data loss with partitions in a AP system like RabbitMQ?

Re: Redis 4.0

#138

Post by [deleted] »

actuator wrote: post #119
Hates_ wrote: post #58

Anyone know when we might see this on AWS (Elasticache)?

I don't like the Elasticache offering of Redis at all. It is severely limited if you are not just using Redis as a cache. There is no way to scale up/down the size of the node without putting it down(this is possible if you run it on EC2 as well as you have access to commands related to replication that are not allowed on Elasticache). We have had master restarting instead of failing over to slave and in the process…

[deleted]

Re: Redis 4.0

#139

Post by zedpm »

Others wrote: post #137
zedpm wrote: post #127

Earlier quoted context omitted.

One less service to manage, plus RabbitMQ's options for dealing with network partitions are all terrible. If you want to run a RabbitMQ cluster, you don't have any option that avoids data loss.

Can you even avoid data loss with partitions in a AP system like RabbitMQ?

RabbitMQ is not strictly an AP system. Its clustering functionality supports several different modes for handling partitions (all of which allow for data loss) and is CP. Federation, which you should use if your network between nodes isn't super reliable, is AP.[0]

What I'd like to see is clustering with support for merging data between nodes. You'd probably end up with duplicates, which I'm OK with, but you'd avoid the data loss that happens with other modes like Pause Minority, where messages get dropped on rejoining a cluster.

[0]: https://www.rabbitmq.com/distributed.html

Re: Redis 4.0

#140

Post by alvil »

numbsafari wrote: post #5

I really wish Redis wouldn't use the "master/slave" terminology. Primary/replica or even Master/replica are better terms. I know I'll probably get flamed for saying this, but it's a pox on a project that I otherwise very much love.

may I use color names as values? is black and white allowed?? is this correct politically?
Post reply on HN