Live data from Hacker News

50 years later, is two-phase locking the best we can do?

concurrencyfreaks.blogspot.com

71–80 of 87 posts

Re: 50 years later, is two-phase locking the best we can do?

#71
post #64

We seriously need to have a new HN policy that requires every link posted to be HTTPS link

Just run Firefox in HTTPS only mode. [0]

You'll get a warning for any site that CAN'T be upgraded to HTTPS, but any site that supports both you'll just go straight to the HTTPS version.

[0] https://support.mozilla.org/en-US/kb/https-only-prefs

Re: 50 years later, is two-phase locking the best we can do?

#72
You could simply add randomized queues.

Example, typically have 1000 tasks and 10..100 hardware threads.

So, make one ordered list of 1000 tasks, and than make copy of it for each thread, each time randomize order of copy.

Than, each thread will just read it's list and execute task and subscribe to one list implemented as non-blocking multi-thread queue.

In worst case, few threads will do some task repeatedly.

And atomic operations this way will scale up to 1000 times.

Re: 50 years later, is two-phase locking the best we can do?

#73
post #62

I'm a beginner in this topic and I find this topic interesting. I really want there to be an easy-to-deploy consistency solution. If I have a distributed microservice architecture and I want to keep multiple datastores in synchronization or "consistent" what's the industry best practice? A few days ago I was trying to solve the inconsistency problem with "settled timestamps" which is a kind of multiversioning idea ex…

> If I have a distributed microservice architecture and I want to keep multiple datastores in synchronization or "consistent" what's the industry best practice? Not to use a distributed microservice architecture.

This is the answer. But if they are going to, I would recommend a book called "Designing Data-Intensive Applications" by Martin Kleppmann. It is a book with the most clear communication on this topic and others that I have ever read. Even if you have a degree in CS, I think it is eye-opening to see how very complex ideas can be communicated vs the same ideas in textbooks.

Re: 50 years later, is two-phase locking the best we can do?

#74

Earlier quoted context omitted.

It seems to me that while it's foolish to assume we can always improve on something, it's not at all foolish to try to do better (even if the attempts don't succeed). Asking "can we do better than this?" and trying to find a better way is a prerequisite for progress, after all.

And trying to improve something will often result in improving ourselves (by learning).

Yeah, that's very true! Honestly, a lot of programming projects I've done are never going to be of use to anyone (even myself). But I learned from all of them, even the ones which went nowhere. That is valuable in itself.

Re: 50 years later, is two-phase locking the best we can do?

#75
post #51
post #50

Earlier quoted context omitted.

Bitcoin: Am I joke to you? Everyone: yes.

Bitcoin is an interesting exception to the above. Issue is latency.

Latency might be OK. The true price of trustless consensus is proof of work, that is, energy.

Re: 50 years later, is two-phase locking the best we can do?

#76
post #59
post #50

Earlier quoted context omitted.

Bitcoin: Am I joke to you? Everyone: yes.

Isn't bitcoin actually eventual consensus determined by increasing unlikelihood of finding a conflicting sequence of hashes?

Yes, although if the consistency is good enough for _payments infrastructure_, it's probably good enough for just about everything.

Re: 50 years later, is two-phase locking the best we can do?

#77

Earlier quoted context omitted.

ulid is great, I use it a lot. There is also a draft to make a new uuid variant – uuid v7 – that will be very similar to how ulid works. https://www.ietf.org/archive/id/draft-peabody-dispatch-new-u...

I use ULID a lot too and It’s frustrating how close the new spec UUIDs are to it without actually being the same… so I’ve got a bunch of code to modify once Postgres supports generation of the new UUIDs server side without extensions or stored procedures. Relatively painless work, but frustrating since it could have been avoided.

ULIDs technically fall under UUID v8 since they map onto the 128bit space. So it's possible that you can just keep using them and PostgreSQL might have something for v8 specifically.

Re: 50 years later, is two-phase locking the best we can do?

#78
Locking algorithms are great, but before applying them to your problem, I think it is important to take a step back and ponder whether you really need to have lots of threads fighting for the same resources. If you're working with in-memory data structures that is expected, but if you're dealing with an external database or another shared external resource, you may be able to do better.

Often, you can batch the requests and hit the external resource with less concurrency, but larger payloads. If the resource handles well batches, you'll need far less concurrency and locking. For example, if you're working with Postgres, you'll need fewer connections, and you may be able to forego adding PgBouncer, which makes things more complicated.

Granted, batching requests is not something most programming languages are well suited to do. Those that are optimized for high concurrency like Go (channels) and Elixir (processes) can do it well, but for languages that do everything with threads it can be painful.

Re: 50 years later, is two-phase locking the best we can do?

#79

Earlier quoted context omitted.

I use ULID a lot too and It’s frustrating how close the new spec UUIDs are to it without actually being the same… so I’ve got a bunch of code to modify once Postgres supports generation of the new UUIDs server side without extensions or stored procedures. Relatively painless work, but frustrating since it could have been avoided.

ULIDs technically fall under UUID v8 since they map onto the 128bit space. So it's possible that you can just keep using them and PostgreSQL might have something for v8 specifically.

I haven’t had my coffee yet but I’m pretty sure v8 still needs to set the version bits to specify its a v8 UUID, which means there’s 6 bits of data that need to be thrown away to migrate from ULID to a fully conforming UUID.

Re: 50 years later, is two-phase locking the best we can do?

#80
post #64

We seriously need to have a new HN policy that requires every link posted to be HTTPS link

After reading something on concurrent algorithms you come up with this irrelevant observation?

Also, no if an http link is about a good concurrent algorithm, I will read it anyways.

Post reply on HN