Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DiceDB

dicedb.io

Post reply on HN
Print view View original

121–130 of 143 posts

Re: DiceDB

#121

Post by jlengrand »

johnisgood wrote: post #12
bdcravens wrote: post #2

Is there a single sentence anywhere that describes what it actually is?

Looks like a Redis clone. The benchmarks compare it to Redis. Description from GitHub: > DiceDB is an open-source, fast, reactive, in-memory database optimized for modern hardware. Commonly used as a cache, it offers a familiar interface while enabling real-time data updates through query subscriptions. It delivers higher throughput and lower median latencies, making it ideal for modern workloads.

I picked that up purely because of the logo / website palette / name choice combinations. Interestingly, not sure it's a good thing.

Re: DiceDB

#123

Post by edoceo »

bdcravens wrote: post #96
remram wrote: post #21

Earlier quoted context omitted.

This is a lot clearer than any information I found anywhere else. There wasn't any room on your website, README, or docs for this summary?

This is a common enough pattern that it should have a name, where the submitted link isn't clear, but a single comment on HN is.

The Cravens Conjecture

Re: DiceDB

#124

Post by fashion-at-cost »

nebulous1 wrote: post #119
ignoramous wrote: post #103

Earlier quoted context omitted.

> single-writer lock-free algorithm I understand the need for correct lock-free impls: Given OP's description, simply avoiding read mutexes can't be the way to go about it?

I don't use Go. https://go.dev/ref/mem If I'm reading this correctly, they are recommending a lock in this situation. However, they are saying the implementations has two options, either raise an error reporting the race (if the implementation is told to do so), or, because the value being read is not larger than a machine word, reply to the read with a correct value from a previous write. If true then it cannot repl…

The goalposts have been moved. The claim is that this pattern isn’t suitable for production code. The ground truth is that a compliant Go implementation may elect to: crash; read the first value ever set to the variable for the entire lifetime of the program; or behave completely as you’d expect from a single core interleaved execution order. The first is an opt-in, the latter two are up to the whims of the runtime and an implementation may alternate between them at any point.

Is that the kind of uncertainty you want in your production systems? Or is your only requirement that they don’t serve “corrupt” data?

Don’t be “clever”. Use locks.

Re: DiceDB

#125

Post by kiitos »

fashion-at-cost wrote: post #124
nebulous1 wrote: post #119

Earlier quoted context omitted.

I don't use Go. https://go.dev/ref/mem If I'm reading this correctly, they are recommending a lock in this situation. However, they are saying the implementations has two options, either raise an error reporting the race (if the implementation is told to do so), or, because the value being read is not larger than a machine word, reply to the read with a correct value from a previous write. If true then it cannot repl…

The goalposts have been moved. The claim is that this pattern isn’t suitable for production code. The ground truth is that a compliant Go implementation may elect to: crash; read the first value ever set to the variable for the entire lifetime of the program; or behave completely as you’d expect from a single core interleaved execution order. The first is an opt-in, the latter two are up to the whims of the runtime a…

Yep. And even if you were to lock down the implementation of the compiler, the version of Go you're using, the specific set of hardware and OS that you build on and deploy to, and so on -- that still doesn't indemnify you against arbitrary or unexpected behavior, if your code violates the memory model!

Re: DiceDB

#126

Post by senderista »

ignoramous wrote: post #103
senderista wrote: post #98

Earlier quoted context omitted.

Haven't looked at the code, but enforcing mutual exclusion between writers but not readers can make sense for a single-writer lock-free algorithm.

> single-writer lock-free algorithm I understand the need for correct lock-free impls: Given OP's description, simply avoiding read mutexes can't be the way to go about it?

Oh, so cycleMap is a non-threadsafe structure? I don't know golang so I didn't realize this.

Re: DiceDB

#127

Post by Apofis »

arpitbbhayani wrote: post #16
bdcravens wrote: post #2

Is there a single sentence anywhere that describes what it actually is?

Arpit here. DiceDB is an in-memory database that is also reactive. So, instead of polling the database for changes, the database pushes the resultset if you subscribe to it. We have a similar set of commands as Redis, but are not Redis-compliant.

Question, how does DiceDB differ from Redis pub/sub? https://redis.io/docs/latest/develop/interact/pubsub/

Re: DiceDB

#128

Post by nebulous1 »

fashion-at-cost wrote: post #124
nebulous1 wrote: post #119

Earlier quoted context omitted.

I don't use Go. https://go.dev/ref/mem If I'm reading this correctly, they are recommending a lock in this situation. However, they are saying the implementations has two options, either raise an error reporting the race (if the implementation is told to do so), or, because the value being read is not larger than a machine word, reply to the read with a correct value from a previous write. If true then it cannot repl…

The goalposts have been moved. The claim is that this pattern isn’t suitable for production code. The ground truth is that a compliant Go implementation may elect to: crash; read the first value ever set to the variable for the entire lifetime of the program; or behave completely as you’d expect from a single core interleaved execution order. The first is an opt-in, the latter two are up to the whims of the runtime a…

I don't disagree, but that's not the claim I was replying to. The question I was asking about was

> I understand the need for correct lock-free impls: Given OP's description, simply avoiding read mutexes can't be the way to go about it?

I did note that the documentation recommends a lock.

> read the first value ever set to the variable for the entire lifetime of the program

That is not my reading of the current memory model? It seems to specifically prohibit this behaviour in requirement 3:

> 2. w does not happen before any other write w' (to x) that happens before r.

Re: DiceDB

#129

Post by fashion-at-cost »

nebulous1 wrote: post #128
fashion-at-cost wrote: post #124

Earlier quoted context omitted.

The goalposts have been moved. The claim is that this pattern isn’t suitable for production code. The ground truth is that a compliant Go implementation may elect to: crash; read the first value ever set to the variable for the entire lifetime of the program; or behave completely as you’d expect from a single core interleaved execution order. The first is an opt-in, the latter two are up to the whims of the runtime a…

I don't disagree, but that's not the claim I was replying to. The question I was asking about was > I understand the need for correct lock-free impls: Given OP's description, simply avoiding read mutexes can't be the way to go about it? I did note that the documentation recommends a lock. > read the first value ever set to the variable for the entire lifetime of the program That is not my reading of the current memor…

In this context, ”happens before” is not a wall-clock colloquialism but in fact a term of art that is specifically described as:

> The happens before relation is defined as the transitive closure of the union of the sequenced before and synchronized before relations.

Without synchronization, the degenerate sequencing is perfectly valid.

That’s the problem with being “clever” - you miss a definition and your entire mental model is busted.

Re: DiceDB

#130

Post by kiitos »

nebulous1 wrote: post #119
ignoramous wrote: post #103

Earlier quoted context omitted.

> single-writer lock-free algorithm I understand the need for correct lock-free impls: Given OP's description, simply avoiding read mutexes can't be the way to go about it?

I don't use Go. https://go.dev/ref/mem If I'm reading this correctly, they are recommending a lock in this situation. However, they are saying the implementations has two options, either raise an error reporting the race (if the implementation is told to do so), or, because the value being read is not larger than a machine word, reply to the read with a correct value from a previous write. If true then it cannot repl…

> However, they are saying the implementations has two options, either raise an error reporting the race (if the implementation is told to do so), or, because the value being read is not larger than a machine word, reply to the read with a correct value from a previous write.

The spec says

> A read r of a memory location x holding a value that is not larger than a machine word must observe some write w such that r does not happen before w and there is no write w' such that w happens before w' and w' happens before r. That is, each read must observe a value written by a preceding or concurrent write.

These rules apply only if the value isn't larger than a machine word. Otherwise,

> Reads of memory locations larger than a single machine word ... can lead to inconsistent values not corresponding to a single write.

The size of a machine word is different depending on how a program is compiled, so whether or not a value is larger than a machine word isn't know-able by the program itself.

And even if you can assert that your program will only be built where a machine word is always at least of size e.g. uint64, the spec only guarantees that unsynchronized reads of a uint64 will return some previous valid write, it doesn't guarantee anything about which value is returned. So `x=1; x=3; x=2;` concurrently with `print(x); print(x); print(x)` can print `1 1 1` or `3 3 3` or `2 1 1` or `3 2 1` and so on. It won't return a corrupted uint64, but it can return any prior uint64, which is still a data race, and almost certainly useless to the application.

Post reply on HN