Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The end of a myth: Distributed transactions can scale

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

81–90 of 95 posts

Re: The end of a myth: Distributed transactions can scale

#81

Post by spenczar5 »

q3k wrote: post #66

The important parts: > The data is assumed to be randomly distributed to memory nodes in the shared memory pool. Any memory node is equi-distant to any compute node, and a compute node needs to reach multiple memory nodes for transaction execution. and also: > Once a memory server fails, NAM-DB halts the complete system and recover all memory servers to a consistent state from the last persisted checkpoint. The recov…

> The world of (actually!) distributed database design has significantly moved on since then.

What are some of the significant movements since then?

Re: The end of a myth: Distributed transactions can scale

#82

Post by necubi »

HopenHeyHi wrote: post #9
jongjong wrote: post #3

That sounds really good. I think reasons for slow adoption are probably a mix of: 1. Lack of developer awareness. 2. Security implications (or perceived implications) of exposing memory directly to a network without passing through CPU or application-level access control mechanisms. The second point may be prohibitive for a lot of general purpose database systems which are intended to be run on shared infrastructure…

The experimental setup involves using a cluster of 56 machines connected by an InfiniBand FDR network That bit might have something to do with it.

Amazon will rent you infiband-class machines like the C6in.metal (https://instances.vantage.sh/aws/ec2/c6in.metal) with 200Gb/s of bandwidth. With EFA (https://aws.amazon.com/hpc/efa/) you can use HPC features like rDMA.

Re: The end of a myth: Distributed transactions can scale

#83

Post by ergonaught »

andruby wrote: post #68
ergonaught wrote: post #63

Earlier quoted context omitted.

[flagged]

Why not? Is the name that bad?

I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database, despite personally liking and respecting the developer.

Simple and honest answer to the question and I have probably the most downvotes I've ever gotten for anything. Y'all are just plain broken.

Re: The end of a myth: Distributed transactions can scale

#84

Post by Supermancho »

ergonaught wrote: post #83
andruby wrote: post #68

Earlier quoted context omitted.

Why not? Is the name that bad?

I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database, despite personally liking and respecting the developer. Simple and honest answer to the question and I have probably the most downvotes I've ever gotten for anything. Y'all are just plain broken.

You: > I don't discuss it because of its name

Also you: > I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database

> probably the most downvotes I've ever gotten for anything. Y'all are just plain broken.

You didn't add to the conversation, except adding your own personal problems as noise. This is working as intended. Hopefully this information will be helpful to all of us, going forward.

Re: The end of a myth: Distributed transactions can scale

#85

Post by ergonaught »

Supermancho wrote: post #84
ergonaught wrote: post #83

Earlier quoted context omitted.

I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database, despite personally liking and respecting the developer. Simple and honest answer to the question and I have probably the most downvotes I've ever gotten for anything. Y'all are just plain broken.

You: > I don't discuss it because of its name Also you: > I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database > probably the most downvotes I've ever gotten for anything. Y'all are just plain broken. You didn't add to the conversation, except adding your own personal problems as noise. This is working as intended. Hopefully this information will be he…

Let's try this again.

Question is posed: What is the explanation for X?

Answer is provided: Here is my explanation for X.

You submit that this answer is "adding personal problems as noise" and downvoted, so "working as intended".

If the relevance leap is hard for you, let's try this.

Question: We just launched a new product called LAKJfkdshfdskjfwfdsfnbozg, but no one is talking about it. Why doesn't anyone ever talk about LAKJfkdshfdskjfwfdsfnbozg?

Answer: I don't talk about it because I don't like the name.

Your response to this would be to downvote the answer because it is a personal problem, noise, which does not contribute to the conversation?

I hope you do not function in any capacity which requires understanding of sentiment.

Re: The end of a myth: Distributed transactions can scale

#86

Post by Supermancho »

ergonaught wrote: post #85
Supermancho wrote: post #84

Earlier quoted context omitted.

You: > I don't discuss it because of its name Also you: > I can't say whether the name is bad or not. I dislike it, and I dislike it strongly enough to not discuss the database > probably the most downvotes I've ever gotten for anything. Y'all are just plain broken. You didn't add to the conversation, except adding your own personal problems as noise. This is working as intended. Hopefully this information will be he…

Let's try this again. Question is posed: What is the explanation for X? Answer is provided: Here is my explanation for X. You submit that this answer is "adding personal problems as noise" and downvoted, so "working as intended". If the relevance leap is hard for you, let's try this. Question: We just launched a new product called LAKJfkdshfdskjfwfdsfnbozg, but no one is talking about it. Why doesn't anyone ever talk…

Let's not. GL with whatever.

Re: The end of a myth: Distributed transactions can scale

#87

Post by eternalban »

mindwok wrote: post #40

If we're bypassing TCP/IP, does that mean we need to build our own protocol on top of this to achieve the reliability guarantees etc provided by the TCP stack?

-- extracted from: https://research.cs.cornell.edu/projects/Quicksilver/public_... --

RDMA (remote direct memory access) is a zero-copy communication standard.

RDMA is a user-space networking solution, accessed via queue pairs: lock-free data structures shared between user code and the network controller (NIC), consisting of a send queue and a receive queue.

RDMA supports several modes of operation [such as] reliable two-sided RDMA operations, which behave similarly to TCP. With this mode, the sender and receiver bind their respective queue pairs together, creating a session fully implemented by the NIC endpoints.

Once a send and the matching receive are posted, the data is copied directly from the sender’s memory to the receiver’s designated location, reliably and at the full rate the hardware can support.

A completion queue reports outcomes. End-to-end software resending or acknowledgments are not needed: either the hardware delivers the correct data (in FIFO order) and reports success, or the connection breaks.

Re: The end of a myth: Distributed transactions can scale

#88

Post by doh »

giovannibonetti wrote: post #57
_448 wrote: post #55

Earlier quoted context omitted.

Why not many are talking about RonDB? This is like Erlang of the database world! People at Ericsson were really very smart and ahead of their time. FoundationDB, TiKV, CitusData's Postgres extension are very well known, but RonDB looks like a hidden gem. What are the practical issues with RonDB that it is not widely known(or used?)?

Because RonDB is not a SQL database, it is just a KV store.

FoundationDB is not a SQL database either though.

Re: The end of a myth: Distributed transactions can scale

#89

Post by [deleted] »

spenczar5 wrote: post #81
q3k wrote: post #66

The important parts: > The data is assumed to be randomly distributed to memory nodes in the shared memory pool. Any memory node is equi-distant to any compute node, and a compute node needs to reach multiple memory nodes for transaction execution. and also: > Once a memory server fails, NAM-DB halts the complete system and recover all memory servers to a consistent state from the last persisted checkpoint. The recov…

> The world of (actually!) distributed database design has significantly moved on since then. What are some of the significant movements since then?

[deleted]

Re: The end of a myth: Distributed transactions can scale

#90

Post by twhitmore »

Haven't read the full paper, but several aspects of this sound less than fully reliable.

It reminds me specifically of 90's multi-client LAN database systems (dBase, Clipper) where clients coordinated via file locks. Unreliability & hangs became a big problem for us.

In the summarized RDMA database, I'd be pretty concerned about reliability & integrity:

1) Crashed servers will leave records locked, and the system will hang. 2) Question whether lock timeouts can be adjudicated reliably. 3) Any errors in server behaviour can easily & widely corrupt data across any other nodes. 4) Overall the RDMA coordination makes me cautious. Can we really replace Paxos with RDMA reliably? If not, problems squeeze out elsewhere. 5) Proposed single-threaded recovery procedure sounds a hazardous operational bottleneck. 6) I'm also cautious about coordination requirements around recovery/ or to transact knowing that recovery is not in process, unless we can show that can be reliable & not add cost to the protocol.

Post reply on HN