Earlier quoted context omitted.
I guess it would be interesting if you re-wrote your website in PHP and wrote a blog post about it with the title 'Why I Moved From Ruby to PHP?'
Or just build a proper Ruby implementation
wrap PHP in a Ruby runtime?
The End of the Redis Adventure
291–300 of 308 posts
Re: The End of the Redis Adventure
#292So they're moving to a new "community based", "light governance" model. [1] There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too. Redis is unique in my experience in that it works the way you'd expect - it doesn't cause outages, it is fast, and it has a vanishingly small number of gotchas. The feature set is well curated and for the most part fits together cohesively. T…
> The most important thing Antirez did, in my opinion, was to say "No" to things. This is hard. It makes you feel bad inside to tell people "No", especially when they open PR and clearly did a lot of work adding some feature.
And some people take the 'No' badly. It's frustrating having to explain that the code in a PR needs to be maintained forever, and it's the repo maintainer that will have to do it, not the contributor. A simple proposal upfront makes all the difference, and saves human effort and feelings.
Re: The End of the Redis Adventure
#293Earlier quoted context omitted.
> There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too. BDFL governance is extremely underrated in the opensource world. People attribute the success of the Linux kernel to its open contribution style, but I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO. Community base…
Underrated in the open source world — whilst at the same time, feels obvious in the writing world? Imagine an author has gotten started writing volumes 1 and 2, and those volumes became popular — maybe the Harry Potter books as an example. Now, adding more authors with equal "story-line decision making power", or a committee, for volumes 3, 4, 5, is a weird idea, right.
Look at what happened with Game Of Thrones! Total disaster at the end that ruined the whole thing for everybody.
Re: The End of the Redis Adventure
#294Earlier quoted context omitted.
Redis is simple. Good. Has a nice api. Has good libraries. Single threadsed. Extremely hard to scale. Impossibly difficult to cluster in containers because it uses hard coded ips to address nodes. Performs poorly with large payloads. Doesn't run on windows properly. Is extremely expensive as a hosted service (orders of magnitude in some cases, eg. azure). You'll love it until you don't. The scaling and clustering sto…
Extremely hard to scale in what sense? A single redis server running on fairly commodity hardware can serve some pretty intense loads (10s of thousands of ops/sec). Once "scale" bigger than that is an issue, companies should already have in place the development staff that will ensure scaling up is done in a sane way, i.e., that have enough sense and/or experience to choose the correct tools for the job. Redis is one…
> Extremely hard to scale in what sense? A single redis server running on fairly commodity hardware can serve some pretty intense loads (10s of thousands of ops/sec).
~90k/sec on a single core of L5520. Scaling Redis is a premature optimization for 99.99% of the use cases. While it may be sexy to talk about millions of ops per second that one's project does in reality the number of projects that have that as a requirement is probably barely in triple digits globally.
Re: The End of the Redis Adventure
#295a very nice interview (in Italian) published last May:
https://www.youtube.com/watch?v=GgT6rXytKds
Towards the end he anticipates what is now public.
Re: The End of the Redis Adventure
#296Earlier quoted context omitted.
I think the mixed historical record of consul republics might be relevant to why a single leader is often a better idea than 2-3.
I'm thinking that a project that starts out with 3 BDFLs (like yours, @albertzeyer) and works fine & builds sth people "love" — it'll continue working fine with those 3 BDFLs. And a project that starts with just 1 BDFL, and works fine & builds sth people "love", it'll continue working fine with that 1 BDFL. If, however, replacing one of those 3, or adding another BDLF to a one person project — I'd think that by defau…
Thing is, with a consul or triumvirate system (you can’t have more than one “dictator”; all of these terms are from Ancient Rome anyway so let’s use them properly), if any disagreement arises between the 2 or 3 leaders, it will develop a new instability where there wasn’t one before. Or else, one of the 2-3 leaders will start to dominate the others. Even without turnover the dictator system is more stable. In fact the only benefit of a consul or triumvirate system might be the stability and continuity under turnover.
Re: The End of the Redis Adventure
#297Earlier quoted context omitted.
A (software) dictator is not necessarily good. But let me double down and say that a community cannot create a product as good as a single head at the top can do, for the single reason that people care only about their own garden, and integration needs someone to oversee and be responsible for all parts. There is no Steve Jobs in a democratic committee.
Cannot seems rather strong. I haven't been paying very much attention, but I was under the impression that Postgres doesn't have a dictator?
It's evolved. There was a point that Tom Lane was the last line of defense against anything that strayed too far.
Heck at one of Postgres conferences there was a shirt that said something like "Tom Lane reviewed my patch and all I got was this stupid shirt."
While he was not a BDFL style person, in the past, if he didn't like a change, it didn't make it in.
Re: The End of the Redis Adventure
#298Earlier quoted context omitted.
We've done just that. Some folks at work wanted sentinel in front as a cluster management layer, but our particular use case did not work well there. Instead, we have some "logical clusters" of 3 nodes that we replicate reads and writes to, sharded to (currently) six clusters. Some logic around quorum for ensuring writes make it to at least 2/3 nodes in a given cluster, with some optimizations for reads sometimes onl…
@sagichmal "... scale ... implement ... sharding ..." @sethammons "... sharded to (currently) six clusters ..." I wonder, when did you start sharding? How many GB memory did your previously single-machine-Redis-instance use, before it was time to shard? How much memory does each "sharded node" have? (It wasn't the CPU (Redis single threaded) that forced you to shard? But because you needed more memory? (Or sth else?)…
> When did you start sharding? How many GB memory did your previously single-machine-Redis-instance use . . .
Sharding is always something you do on day 1 for every project, because a single-machine-Redis-instance is a SPOF and won't pass even the most basic operational readiness checklist.
> (capacity planning questions)
The answers to these questions don't generalize, they depend on your workload. You should figure out some approximation of your data types and request profiles, and use those to get a rough understanding of what one Redis server on a given machine class is capable of delivering. This usually means applying a few different classes of read/write loads against a small cluster at steadily increasing RPS, and documenting how CPU, memory, and latency characteristics change. From these numbers it's possible to derive a capacity plan.
The last time I did this, for my workload and machine class, one Redis instance (one core) delivered 250-500k RPS, invariant to memory used. We'd use the conservative end of that range, combined with the RPS and data set growth rates we predicted, to provision for ~12 months of growth. Operationally, we would deploy (cores-1) or (cores-2) instances per host (I forget exactly) on 32- and 64-core machines. I think they had like 64-128G of RAM, and we made sure to leave enough memory overhead so the AOF or whatever persistence option wouldn't lock up the box. But even the choices of what class of machines to use is a function of your use case, if you have really large dataset with relatively non-costly (CPU) operations, you want a totally different machine profile than a relatively small dataset with complex operations. Availability SLOs also factor in.
All of this is basic operational stuff, and with Redis the answers are pretty well-understood and highly predictable. Completely opposite to systems like Elasticsearch, which was a total nightmare to predict, provision for, and operate.
Re: The End of the Redis Adventure
#299Earlier quoted context omitted.
Does FreeBSD have a BDFL? We don't have enough data points to determine if what you are saying is correct. I have also seen lots of BDFL projects smoke out. Is the lesson that actual dictatorships only survive long term if 1) the dictator is a nice person and 2) they eventually get managed by an open democratic process? I think you are actually describing the two clocks problem. Lots of BDFL projects have their own C…
this is IMHO no different the real world - a 'good' despot/monarch/etc can be a good leader, but this is highly dependent on the situation (read: leader), and leadership transition process is always a risky one. Democracies/Republics work well so long as the community has a generally shared vision, but can be hampered when there is too much dissent, but generally avoid the succession problem
You can fork Linux if the dictator doesn’t suit you, but you can’t fork a nation. You also can’t switch to another nation that is more appealing, generally. Finally, Linux can’t imprison and torture you if you criticize it
Re: The End of the Redis Adventure
#300Earlier quoted context omitted.
@sagichmal "... scale ... implement ... sharding ..." @sethammons "... sharded to (currently) six clusters ..." I wonder, when did you start sharding? How many GB memory did your previously single-machine-Redis-instance use, before it was time to shard? How much memory does each "sharded node" have? (It wasn't the CPU (Redis single threaded) that forced you to shard? But because you needed more memory? (Or sth else?)…
> When did you start sharding? How many GB memory did your previously single-machine-Redis-instance use . . . Sharding is always something you do on day 1 for every project, because a single-machine-Redis-instance is a SPOF and won't pass even the most basic operational readiness checklist. > (capacity planning questions) The answers to these questions don't generalize, they depend on your workload. You should figure…
> "Sharding is always something you do on day 1 for every project ... SPOF ..."
Cannot agree with that. If one does that or not, would depend on things like Service Level Agreements (SLA) — and one can have a pretty high SLA uptime %, without sharding, e.g. if there's an underlying pretty stable hosting provider that live migrates if there's a hardware failure.
Thanks for writing about the last time you used Redis. Interesting to hear that, in that case, the machines had sth like 64 – 128 RAM, and 250k – 500k RPS. Yes I agree that I'd need to benchmark and think about what type of machine(s) to use (some time later — a bit too early for that now). Sounds as if you are / were a pretty large company / project, needing that much memory and machines :- )