51–60 of 169 posts
Looking through the code it supports fuse, but the documentation in ENG is sparse. It also looks to underpin Tera: the Baidu distributed DB. I think a low read/write latency dfs suitable for real time applications would be a game changer. I'm hoping they up the documentation from here and engage the English speaking community.
More like a C++ clone of HDFS than most people are likely hoping. While you seem to be able to mount it with FUSE I imagine it's primarily meant to be programmed against directly. Using Raft over a dependency on an external consensus system is nice. Definitely makes the namenode architecture much better.
Another important aspect is that is using SSD + SATA(I suppose) , which could be a better option than standard SATA/SSD or LV cache using SATA + SSD.
Even if it's just a new thing, if it proves to be faster it may be implemented in Hadoop ecosystem in the future. HDFS has a lot of features being a mature piece of software but it lacks on the response time.
Why is there no English documentation ?
Looks like a pretty good first attempt at a distributed filesystem. Initial impression is HDFS with a distributed NameNode/Nameserver. The first diagram also shows a Metaserver layer that's not mentioned at all in the more recent of the two design docs but "separate Metaerver from Nameserver" appears (unchecked) in the roadmap. All operations using access methods other than their own SDK seem to get funneled through the NameServer cluster, which will severely limit throughput. Not clear how they do replication, though weakly implied that it's driven from the client (like Gluster) or NameServer rather than the first ChunkServer (like Ceph, HDFS, everything else). No mention of how they handle consistency or repair. Likewise no information about performance or security. Not clear if it's anywhere near POSIX compliant (probably not).
FUSE support is in the diagrams, but not checked off on the roadmap. Slow-node detection and avoidance seemed like one of the most interesting features from the design, but is not checked off either. Other things not even on the roadmap, using Gluster not as a fair comparison but as a handy list of possibilities: multiple replication levels, tiering, erasure coding, NFS/SMB, caching, quota, snapshots.
As I said, looks like a good first attempt. Better than most I've seen, with lots of potential, but as of today it seems rather bare-bones. Many hard problems remain to be solved, and I wish them well.
Based on the design[1], it has a leader / follower pattern (although you should have multiple leaders with Raft consensus to avoid having a single point of failure), where the leader is called "nameserver" and decides where to put each piece of data and metadata among a set of chunk servers and metadata servers. That design is very reminiscent of CephFS's cluster monitors, metadata servers, object storage devices. [1…
I thought Raft was always-single-leader? Followers can happily become leaders through elections?
Earlier quoted context omitted.
Why is there rarely any non-english-language documentation for most codebases nowadays? The world doesn't revolved around english-speaking countries.
Because English is the lingua franca of the software industry, and developers are usually expected to know English, no matter where they're from.
Earlier quoted context omitted.
Not to be rude, but I'm glad I don't work where you work. How would that be consistent with your own classes? "Oh no, you can't just use a 'Tree' object, you need to explicitly set that there are no leaves yet, no branches yet, no squirrels yet, etc… etc…" Do you .clear() your vectors before you use them? This sounds like newbies that do: #define TRUE (1 == 1)
It's not about initialization, it's more about specifying clearly what happens in all cases. Anyway the reason for which it would no pass review is that today you use one compiler, tomorrow you have to use another and then you have to review all these little details again. It's about saving money more than anything and you do that by not relying on compiler behavior.
Good thing then that it's mandated by the language reference, and not up to the compiler to decide. According to C++11, §21.4.2/1, an uninitialized std::string should be an object of class std::basic_string with non-null data and a size of 0.
Earlier quoted context omitted.
Not to be rude, but I'm glad I don't work where you work. How would that be consistent with your own classes? "Oh no, you can't just use a 'Tree' object, you need to explicitly set that there are no leaves yet, no branches yet, no squirrels yet, etc… etc…" Do you .clear() your vectors before you use them? This sounds like newbies that do: #define TRUE (1 == 1)
It's not about initialization, it's more about specifying clearly what happens in all cases. Anyway the reason for which it would no pass review is that today you use one compiler, tomorrow you have to use another and then you have to review all these little details again. It's about saving money more than anything and you do that by not relying on compiler behavior.
If a different compiler breaks this behavior, it's not standard compliant and thus could do all sorts of stuff in every possible line, including in:
std::string pad = "";