Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis array: short story of a long development process

antirez.com

Post reply on HN
Print view View original

51–60 of 118 posts

Re: Redis array: short story of a long development process

#51

Post by epolanski »

tibbar wrote: post #47

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

Postgres and Redis are dramatically different projects with radically different stories, contributions and development team.

Virtually all major Redis features are a solo job of the post author.

By the way reviewers are paid good money for this and know the setup.

Re: Redis array: short story of a long development process

#52

Post by epolanski »

gurgeous wrote: post #35

Thanks for adding this. Excited about array/regex, also very interested in your experience using LLMs to stretch your abilities. There are many of us laboring quietly on various projects attempting the same. "Vibe coding" (and the backlash) doesn't really capture how we work.

The problem with "vibe coding" is that the author who coined the term gave it a very specific definition (after all, it's his term): writing software without looking at the code, just "vibing".

Then it quickly lost its original meaning as people started using it for virtually all forms of AI-assisted coding.

Re: Redis array: short story of a long development process

#54

Post by antirez »

ericpauley wrote: post #46

Couldn't some of the use cases presented for this be accomplished with ZSETs? I get the performance angle, but it seems that this could have been accomplished without the new API surface by selectively optimizing ZSET storage for dense values (in the same way that Arrays selectively use sparse representations). The RE component is interesting, but as commentary here has noted it seems orthogonal to the array data str…

Unfortunately not, sorted sets are actually a bit in the other side of the spectrum: they are semantically sound, but absolutely wasteful because of the combined skiplist + array. Also, if the underlying representation is not an array, range queries and ring buffers will never be as efficient and compact as they should. In theory you can do everything with everything, but segmenting what each API can do allows you to exploit the use cases to provide the best underlying implementation.

Re: Redis array: short story of a long development process

#55

Post by tibbar »

epolanski wrote: post #51
tibbar wrote: post #47

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

Postgres and Redis are dramatically different projects with radically different stories, contributions and development team. Virtually all major Redis features are a solo job of the post author. By the way reviewers are paid good money for this and know the setup.

Oh wow, I didn't realize that Redis is still mostly just authored by antirez! (My understanding is that he had left for some time and then returned to the project.) That is, honestly, pretty amazing. Well, redis is great and clearly it's worked out.

Re: Redis array: short story of a long development process

#56

Post by antirez »

tibbar wrote: post #47

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

The code is 5000 lines of code in total, comments included:

2000 lines the sparse array.

2000 lines the t_array commands and upper layer implementation.

~500 lines of AOF / RDB code.

All the other stuff is tests, JSON command descriptions, TRE library under "deps".

Re: Redis array: short story of a long development process

#57

Post by pbowyer »

lovasoa wrote: post #25
wood_spirit wrote: post #20

Sharing my current MO: I start with a high level design md doc which an AI helps write. Then I ask another AI - whether the same model without the context, or another model - to critique it and spot bugs, gaps and omissions. It always finds obvious in hindsight stuff. So I ask it to summarize its findings and I paste that into the first AI and ask its opinions. We form an agreed change and make it and carry on this a…

How much faster/slower are you with that process compared to writing code yourself?

Developer of 20+ years here, can't give you an accurate multiplier but I am faster.

Because spotting holes in specs has never been one of my strengths. And working without technical colleagues much of the time, it's a boon to be able to "rubber-duck" my ideas with something that is at least more intelligent than plastic.

Grabbing multipliers from thin air, the coding bit may only be 2x faster with a poorer-quality outcome, but working out what's needed is a good 5x faster.

And yes, I'm using the same adversarial AI MO as @wood_spirit, combined with Matt Pocock's excellent /grill-me and /grill-with-docs skills [1] and Plannotator [2] to review the plans.

1. https://github.com/mattpocock/skills

2. https://github.com/backnotprop/plannotator

Re: Redis array: short story of a long development process

#58

Post by fancy_pantser »

antirez wrote: post #56
tibbar wrote: post #47

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

The code is 5000 lines of code in total, comments included: 2000 lines the sparse array. 2000 lines the t_array commands and upper layer implementation. ~500 lines of AOF / RDB code. All the other stuff is tests, JSON command descriptions, TRE library under "deps".

I think the point GP is making is this is a PR that smells like a solo dev working on their own project and not how a community-driven project adds major new functionality, although I'm sure there are docs and descriptions (or at least a discussion of tradeoffs and design decisions if not ADRs) are somewhere, but not linked handily to the PR. There is a lot of explanation in the blog post and PR, but it's unilateral-looking.

c.f. valkey and others

Re: Redis array: short story of a long development process

#59

Post by antirez »

epolanski wrote: post #26

Got few questions: - the project essentially spans almost 3 different (albeit minor) generations of LLMs. Have you noticed major differences in their personas, behavior, output for that specific use case? - when using AI for feedback, have you ever considered giving it different "personalities"? I have few skills that role play as very different reviewers with their own different (by design conflicting) personalities…

1. The huge jump from from Opus to GPT 5.3. Game changer. GPT 5.4, 5.5, were better but only incrementally better.

2. Nope I don't give much personalities, but I use subtle prompt differences to maximize certain responses I want, to make the model focusing in a given detail or acting in a specific kind of engineering mindset.

3. It never happened that the AI was slowing me down since I always had the full context and code detail in mind of what was happening. I believe that this happens more when you don't have a clear idea. Also GPT >= 5.3/4 is not the past generation of models, it is very hard to trap it into a situation where it seems unable to understand what you mean.

4. A few times the AI provided fresh insights that I really liked. Most of the times it was the other way around. Certain implementations were written by the AI at a very impressive level of quality.

5. I don't use general skills, I build skills with deep search when needed for specific projects, and build an AGENT.md that works as a knowledge base as I work with the AI. One thing that I use a lot is, when there is a very complex problem, to tell GPT that I have a friend called Machiavelli that is an incredible computer scientist. To write him an email in /tmp/letter.md with the problem we are facing, and I'll try to get a reply. Then I ask GPT 5.5 Pro on the web with extensive reasoning set on. It will take sometimes 30 minutes or more to reply. Often times after I feed back the reply, the agent will be able to see things a lot more clearly.

Re: Redis array: short story of a long development process

#60

Post by tibbar »

antirez wrote: post #56
tibbar wrote: post #47

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

The code is 5000 lines of code in total, comments included: 2000 lines the sparse array. 2000 lines the t_array commands and upper layer implementation. ~500 lines of AOF / RDB code. All the other stuff is tests, JSON command descriptions, TRE library under "deps".

First of all, redis is amazing, and your 4 month development process speaks to the fact that you've already designed and verified correctness super thoroughly.

... just speaking as someone who sometimes has to review very long PRs sometimes, though, I feel like 25% is a roughly normal level of "signal to noise." 5,000 lines of core logic is a LOT, and the tests and dependencies do still need to be read.

EDIT: I feel like the problem, as a reviewer, is processing 4 months of intensive research/development and providing useful feedback. At that point, there's probably not much major input you can have into the core architecture or strategy, so you're probably not providing much more than a bugbot at that point.

Post reply on HN