Live data from Hacker News

Baidu File System – A distributed file system for real-time applications

github.com

161–169 of 169 posts

Re: Baidu File System – A distributed file system for real-time applications

#161

Earlier quoted context omitted.

> Also if GC was such a huge problem, exchanges or HFT companies wouldn't use Java for their low latency stuff, and there definitely are companies which do. Can you name one?

LMAX, New York Exchange.

Wow, that's neat. Thanks for the pointer!

Re: Baidu File System – A distributed file system for real-time applications

#162
post #8

Looking through the code it supports fuse, but the documentation in ENG is sparse. It also looks to underpin Tera: the Baidu distributed DB. I think a low read/write latency dfs suitable for real time applications would be a game changer. I'm hoping they up the documentation from here and engage the English speaking community.

Speaking of documentation, I see almost no comments in the code. It all looks reasonable, but this takes self-documenting to an extreme.

The code is not very self-documenting either. There are >50 line long functions with mixed levels of abstraction, and error handling code is completely mixed with logic. I find such code quite hard to read and lack of comments doesn't help.

Re: Baidu File System – A distributed file system for real-time applications

#163

Why is there no English documentation ?

Hopefully the Chinese will not be as arrogant as you when they take over the leading role on this planet.

I was not arrogant. I am not a native English speaker. I expect most softwares to have a documentation in English.

Do you want to learn 5 different languages just because 5 different kind of people write good software ?

It's easy to blame others.

Re: Baidu File System – A distributed file system for real-time applications

#164
Not commenting on the BDFS so much as its really cool to see large Chinese companies contributing to open source, does anyone know of other large projects outside of the main Android forks? Pardon the ignorance.

Also wonder if there will be larger skepticism toward integrating Chinese O/S in regards to potential influence by the government (like the NSA has tried to influence in the past)

Re: Baidu File System – A distributed file system for real-time applications

#165

Earlier quoted context omitted.

It's the PC version of master/slave. See [0] for the original madness... [0] https://github.com/antirez/redis/issues/3185

antirez handled that pretty well.

Yeah, I'd say so too. IMHO I think it was a mistake to use the docs for political purposes, mostly because doing so lends legitimacy to issue.

Where I think he did well was in de-escalating tension.

Re: Baidu File System – A distributed file system for real-time applications

#166
For the distributed FS people out there I've got a complicated question. In my job I need to poll and collect data from many remote sensor devices, log all the output, and process that. Not only do I do this buy MANY of my colleges do this and have a different way to manage this process. Can a distributed file system help with this case?

Is there any file system that would be able to sync what amounts to text/binary data across many hosts and allow me to aggregate the data off the network for more secure storage?

I was thinking about using IPFS for this but this also seems better. I'd hopefully like to have a private network for this use case so that other people can't post up a device on this file system and introduce fake data.

Re: Baidu File System – A distributed file system for real-time applications

#167
post #73

Earlier quoted context omitted.

In Java the default value for a String (or any object) is null, not an empty string.

Did you not read my whole comment? Please read the whole thing before replying.

Read the whole comment. Still don't see nothing that invalidates my answer to "Can you name one language that has strings that initialise to anything but a valid object containing an empty string?".

Re: Baidu File System – A distributed file system for real-time applications

#168

Earlier quoted context omitted.

Did you not read my whole comment? Please read the whole thing before replying.

Read the whole comment. Still don't see nothing that invalidates my answer to "Can you name one language that has strings that initialise to anything but a valid object containing an empty string?".

You specifically mentioned Java, and I specifically mentioned reference-based languages, with Java being the most obvious example.

Re: Baidu File System – A distributed file system for real-time applications

#169

Disclosure: I'm a Gluster developer. Looks like a pretty good first attempt at a distributed filesystem. Initial impression is HDFS with a distributed NameNode/Nameserver. The first diagram also shows a Metaserver layer that's not mentioned at all in the more recent of the two design docs but "separate Metaerver from Nameserver" appears (unchecked) in the roadmap. All operations using access methods other than their…

> Looks like a pretty good first attempt at a distributed filesystem. You are damn right. It is. ~~ 3 years ago, the most widely used DFS in baidu was Peta which is similar to HDFS V2. We have migrate to AFS now.
Post reply on HN