Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Baidu File System – A distributed file system for real-time applications

github.com

Post reply on HN
Print view View original

81–90 of 169 posts

Re: Baidu File System – A distributed file system for real-time applications

#81

Post by Cyph0n »

kuschku wrote: post #79
ergo14 wrote: post #76

Earlier quoted context omitted.

what is the leader/follower pattern? Something like master/slave approach?

Leader/Follower is actually something entirely different, but the linked chinese document talks about a master/client approach. Sadly, in the past years, due to some political movements, the term "master/slave" has been declared problematic, and GitHub actively warns that projects using such language can and will be excluded from the service. There have been previous discussions about this on HN.

Wow, that's very interesting. I thought GitHub delegates moderation to the repo owners.

There was actually a huge debate about this on Reddit caused by Swift merging a rename change PR into master. The Swift team was so excited about the change for some reason that they didn't even run tests before the merge...

Re: Baidu File System – A distributed file system for real-time applications

#82

Post by omginternets »

ergo14 wrote: post #76
espadrine wrote: post #46

Based on the design[1], it has a leader / follower pattern (although you should have multiple leaders with Raft consensus to avoid having a single point of failure), where the leader is called "nameserver" and decides where to put each piece of data and metadata among a set of chunk servers and metadata servers. That design is very reminiscent of CephFS's cluster monitors, metadata servers, object storage devices. [1…

what is the leader/follower pattern? Something like master/slave approach?

It's the PC version of master/slave. See [0] for the original madness...

[0] https://github.com/antirez/redis/issues/3185

Re: Baidu File System – A distributed file system for real-time applications

#83

Post by gnipgnip »

I just want to note how comments complaining about Mandarin being the dev language, gets downvoted (by sympathetic Europeans);

while at the same time so do those complaining about English's hegemony in India (by furious Indians).

Strange is our world.

Re: Baidu File System – A distributed file system for real-time applications

#84

Post by chronid »

merb wrote: post #50

I wonder whey they choose to rewrite raft and didn't use something with etcd or another working raft solution.

You usually don't want to add another dependency you don't control to your system, if you don't have to (in term of time and resources).

It's another point of failure, more infrastructure you have to keep alive...

Re: Baidu File System – A distributed file system for real-time applications

#85

Post by wstrange »

gnipgnip wrote: post #75
FraaJad wrote: post #70

Earlier quoted context omitted.

Why would India use Mandarin? English is the most common script in India, why would we use anything else. Unless you are Hindian trying to impose your minority language on the rest of us.

Wow. Just wow. - English is hardly the "most common script". Just because the retainers in Delhi impose the colonial apparatus on us, doesn't automagically give it "statistical power" as well. - Every state has a (poor, uneducated, illiterate) captive linguistic population more than that of Korea; no reason they ought to use Hindi, nor even Nagari (script != language, in case your education didn't tell you that). - A…

Note the debate you are having with the GP is in English.

For better or worse, English is the common language for software development (and aviation, etc.)

Re: Baidu File System – A distributed file system for real-time applications

#87

Post by notacoward »

usgroup wrote: post #78
lylei wrote: post #51

Earlier quoted context omitted.

Thanks for your advice. We are working on translating all the documents :)

PS: if your DFS works within a docker container you'll have a very strong differentiator since the rest don't. You'd also possibly solve the "how to do storage in a container cloud without resorting to NAS or separate clusters" problem.

> if your DFS works within a docker container you'll have a very strong differentiator since the rest don't.

Untrue. Gluster is already deployed that way in many places. Yes, in production and at scale.

Re: Baidu File System – A distributed file system for real-time applications

#88

Post by gnipgnip »

wstrange wrote: post #85
gnipgnip wrote: post #75

Earlier quoted context omitted.

Wow. Just wow. - English is hardly the "most common script". Just because the retainers in Delhi impose the colonial apparatus on us, doesn't automagically give it "statistical power" as well. - Every state has a (poor, uneducated, illiterate) captive linguistic population more than that of Korea; no reason they ought to use Hindi, nor even Nagari (script != language, in case your education didn't tell you that). - A…

Note the debate you are having with the GP is in English. For better or worse, English is the common language for software development (and aviation, etc.)

> Note the debate you are having with the GP is in English.

You don't say ?

> For better or worse, English is the common language for software development (and aviation, etc.)

English in India is more dense than the feudal castes of medieval Europe; hardly a professional thing this.

The Human-rights wallahs don't complain precisely because the current state of affairs benefits the nations that control them; much as it does their native retainers.

Re: Baidu File System – A distributed file system for real-time applications

#89

Post by erikb »

gnipgnip wrote: post #67

Can't speak much for the project, but have to admire them for sticking with Mandarin. India is atleast a 100-200 years away from something of this kind happening; or more likely never at all. Indeed, there is not a single research university worth its name that isn't also essentially an export hub of brains to the 5-eyes (& Singapore). English is crucial for India's system of feudal slavery to work.

I don't think they stick to Mandarin because they want to. English is considered hip and intellectual in China, especially in the first tier cities which I suppose the developers of such modern technology are. But the problem is that English is really, really hard for Chinese native speakers, since grammar, words, culture and pronunciation are so different from all the Chinese languages.

So as a team leader or project manager in China I would probably also stick with Chinese since it is much easier to find really good and not too expensive employees that way. Let them try to use English, support the ambition, but don't enforce it.

And I don't know much about India but from what I heard is that it is more a cultural issue that India lacks behind. Everything is (so I heard) still very traditional and backward focussed. While China as a country spent 20-30 years to become more open for new ideas and approaches. How true is that from other people's perspective here?

Re: Baidu File System – A distributed file system for real-time applications

#90

Post by rwallace »

gnipgnip wrote: post #83

I just want to note how comments complaining about Mandarin being the dev language, gets downvoted (by sympathetic Europeans); while at the same time so do those complaining about English's hegemony in India (by furious Indians). Strange is our world.

Maybe both kinds of comments are being downvoted by those of us who like technical conversations not to be full of people bitching about other people's choice of language. I don't find that strange at all.
Post reply on HN