21–30 of 42 posts
At this stage, the only thing I care about is which license they are going to apply to this project. That'll be enough for me to know if I should bother investing any further time here.
Earlier quoted context omitted.
We've had many discussions about licensing issues, but this isn't one of them. Please don't post off topic and then complain about downvotes—it breaks the guidelines. https://news.ycombinator.com/newsguidelines.html
> we've had many > don't post off topic Users of HN, please note "we" is Facebook and Facebook, or a representative of Facebook, is telling us what we can and can't discuss here. This is why I quit posting here and use /r/hackernews on Reddit for my links. However, I occasionally make an appearance when I see patterns emerge, such as control where we didn't assume control existed.
Earlier quoted context omitted.
I never imagined that the project's license would be off-topic. To me a project's license is very material, a deciding factor if I'm using it or not. That's what I was trying to express.
The project under discussion doesn't have a license.
Can someone explain/expand on the above please? I've read the article a couple of times and tried to understand the above paragraph in context but I don't get it.
> We continue to iterate on LogDevice with the ultimate goal of contributing it to the open source community later in 2017. Frankly, if it's the same license as React (BSD + PATENTS), I'm not interested. Edit: Here come the facebook fanboys with the downvotes.
We've had many discussions about licensing issues, but this isn't one of them. Please don't post off topic and then complain about downvotes—it breaks the guidelines. https://news.ycombinator.com/newsguidelines.html
Why do you decide what we discuss? Since you squashed my comment about licensing, another one popped up. It means people do think licensing is important to discuss.
The two sound very similar. I've been studying Bookkeeper this week to understand how it works and was excited to see the blogpost about LogDevice.
My understanding isn't fully there yet, but BookKeeper seems to have a more involved protocol versus LogDevice: ie it has fencing to ensure that only one writer is writing to a log at a time a 2-phase-commit-like protocol, and opening/closing of ledgers.
To me LogDevice's sequencer and epoch number sounds much simpler. Does BookKeeper achieve some more consistency or other guarantees that LogDevice doesn't by having a more involved protocol? How do the two compare in terms of goals/pros/cons/tradeoffs/usecases/etc?
(From reading the blog post it seems to me that in LogDevice an old sequencer could still be writing to LogDevice when a new one with a higher epoch number is also writing. As I understand it BookKeeper uses a CAS operation on metadata version number (like the epoch number of LogDevice) AND fencing on the storage nodes to make sure that only a single writer is writing at a time).
>> We ensure that only one copy of every record is read from disk and delivered over the network by including the copy set in the header of every record copy. A simple server-side filtering scheme based on copy sets coupled with a dense copy set index guarantees that in steady state only one node in the copy set would read and delivery a copy of the record to a particular reader. Can someone explain/expand on the abo…
>> We ensure that only one copy of every record is read from disk and delivered over the network by including the copy set in the header of every record copy. A simple server-side filtering scheme based on copy sets coupled with a dense copy set index guarantees that in steady state only one node in the copy set would read and delivery a copy of the record to a particular reader. Can someone explain/expand on the abo…
A copy set is the list of storage node ids that the sequencer chose as recipients for a record. This piece of metadata is stored on storage nodes alongside record payloads in an index that maps the sequence number to copy set mapping.
Storage nodes consult the index to filter out payloads they shall not send to readers. The filtering logic consists in sending the copy if the storage node sees itself as the first recipient in the copy set after it's been shuffled using the client id as a seed.
This results in readers receiving exactly one copy of the payload.
Reading the blog its not clear to me how they deal with gaps in the LSN sequence. The scalability & performance properties derive from 1) Using a separate sequencer that issues increasing sequence numbers, 2) Uncoordinated distributed writes of actual record value to storage nodes, 3) reconstitution of ordered log at consumer side. How does a consumer that have retrieved N and N+2 know if N+1 is not yet written, or i…
Within an epoch, the sequencer is a single process on a single machine and gives out LSNs sequentially. When a sequencer dies, and a new one is started, its first job is to fix up the end of the last epoch. If it can't find any copies of a given record, it inserts a "hole plug," to store the fact that the record is lost. So, except for hole plugs (which should be very rare), the only gaps are between epochs as you say.
Suppose you have 10 LogDevice servers, and you store 3 copies of every record. Then the client will have connections to all 10 machines, and each machine will push whatever records it has to the client. Crucially, the servers always push records in order. So if a client gets record N from machine 7, then gets record N+2, it can be sure that machine 7 doesn't have a copy of record N+1.
Once you've got record N+2 or higher from at least 8 machines, without getting record N+1 from any of them, you can be sure that at least one copy of the record has been lost. If those 8 servers have complete information, you can report to the user that the record has been lost.
>> We ensure that only one copy of every record is read from disk and delivered over the network by including the copy set in the header of every record copy. A simple server-side filtering scheme based on copy sets coupled with a dense copy set index guarantees that in steady state only one node in the copy set would read and delivery a copy of the record to a particular reader. Can someone explain/expand on the abo…
They've neglected to define the term "copy set". I assumed it was some kind of record ID (although I don't get why that should be distinct from the Log Sequence Number).