> Durability: RAMCloud replicates all data on nonvolatile secondary storage such as disk or flash, so no data is lost if servers crash or the power fails. How does this work if someone is doing multiple sequential writes? Doesn't backup-ing to disk take a lot longer than writing to _RAM_ meaning some writes could get lost?
You can distribute writes over multiple disks.
RAMCloud Project
31–40 of 52 posts
Re: RAMCloud Project
#32How does it replicate all the data on disk or flash while maintaining the write latency of DRAM? I'm thinking there must be a delay during which loss of power will result in loss of data that was acknowledged as written. Is this something that RAMCloud overcomes in a novel way, or is the answer simply that the data is replicated across many nodes? How about the problem of raw bandwidth? If you keep sending writes to…
Re: RAMCloud Project
#33Re: RAMCloud Project
#34Earlier quoted context omitted.
You can distribute writes over multiple disks.
Out of pure curiosity but wouldn't the number of disks required to guarantee 100% write-backup at 100% peak times be pretty big? A quick search says HDD is ~200 times slower than RAM (on average, consumer grade).
Re: RAMCloud Project
#35Earlier quoted context omitted.
They just host the site, it is their wiki product called Confluence. The same way Github hosts pages at github.io but has nothing to do with content that people host there.
Oh, I know, I administer Confluence, Jira and Bitbucket. Bar from bitbucket, the other two hosted versions are woefully slow. Don't get me started about the Jira Calendar. If they're blogging about some tech, maybe they need to dogfood that first.. especially when it's due to performance
This is a stanford.edu project wiki page, hosted by Atlassian. Nothing more.
By the same logic, any performance improvement that Github might see from random projects hosted on Github, should be "dogfooded" in the same way.
It's not dogfooding when it's someone else's work.
Re: RAMCloud Project
#36Earlier quoted context omitted.
They just host the site, it is their wiki product called Confluence. The same way Github hosts pages at github.io but has nothing to do with content that people host there.
Oh, I know, I administer Confluence, Jira and Bitbucket. Bar from bitbucket, the other two hosted versions are woefully slow. Don't get me started about the Jira Calendar. If they're blogging about some tech, maybe they need to dogfood that first.. especially when it's due to performance
Let's break this apart a bit more slowly.
They did not blog about this.
Their users did.
Atlassian, the company hosting it, did not write the post.
Their customers, the people using Atlassian's servers, did write the post.
Re: RAMCloud Project
#37Re: RAMCloud Project
#38Re: RAMCloud Project
#39Re: RAMCloud Project
#40A cool tiny bit of trivia about RAMCloud: that's from this project that the Raft consensus algorithm emerged! ( https://raft.github.io/raft.pdf ) Right now, I think that the algo is used in RAMCloud via LogCabin ( https://github.com/logcabin/logcabin ). Raft is more practical (as in "well specified") than Paxos and closest to its lesser known cousin, VR ( Viewstamped Replication ). Beyond the academic genealogy of th…
For one thing, implementing Raft in a performant way is incredibly difficult. Where paxos has very fine-grained units of consensus (e.g., ballots), in Raft, everything happens through leases, and is totally linear. This is one of the major reasons a bunch of (all?) major Raft implementations do unprincipled things like non-quorum reads. This sort of problem also makes it very difficult to get Raft to do anything on an unreliable network. Raft is also super noisy, so you have to do stuff like piggy back heartbeats in other RPC calls.
One way of thinking about this is: in paxos much of the complexity is in the specification itself. Performant Raft pushes this complexity into external systems. Both of these are extremely hard problems. There is no free lunch.
If the problem is really just verifying the model, then I think you are better off writing a simple modeling language, writing paxos in it, and then compiling that to c++ or something. This was the approach Google took in paxos made live[1] for example.
[1]: http://www.read.seas.harvard.edu/~kohler/class/08w-dsi/chand...