Live data from Hacker News

Baidu File System – A distributed file system for real-time applications

github.com

111–120 of 169 posts

Re: Baidu File System – A distributed file system for real-time applications

#111

Earlier quoted context omitted.

They put that code in github, releasing it to non-chinese developers. Won't it be good if code has at least some documentation in English ? It's not that they don't want, they have some part in English.

The hegemony of English is unquestionable.

Last time I checked English was the 3rd most spoken language on the planet.

- Chinese

- Spanish

- English

You mean in tech?

Re: Baidu File System – A distributed file system for real-time applications

#112
post #72
post #18

https://github.com/baidu/bfs/blob/master/src/client/bfs_clie... std::string pad; if (path[path.size() - 1] != '/') { pad = "/"; } Else?

what if path is empty?

Then they have a problem. Undefined Behavior.

Re: Baidu File System – A distributed file system for real-time applications

#113

Earlier quoted context omitted.

"It looks like a faster version of HDFS since it's written in C++ (vs Java)." This is non sequitur. The conclusion does not follow from the premise.

During non-GC periods, probably true. But having a realtime filesystem service that is prone to stop-the-world GC pauses is a showstopper for many applications. Also, a C++ implementation is likelier to use far less memory than a Java implementation, assuming the skills of both programmers are roughly equal.

The underlying local filesystem on each node is not truly realtime, so a "realtime distributed file system" is already quite a stretch. Also JVM is perfectly fine with pause times below a few tens of ms worst-case (when using properly tuned G1, CMS GC), which is lower than worst-case latency induced by network + I/O.

As for using less memory - you don't allocate buffers for file data on the JVM heap. You allocate them in native memory exactly as you'd do it in C++. Therefore it is possible to create a JVM-based file system that handles petabytes of data with just as little as 100 MB heap, used mostly for small temporary objects.

Also, the code here is using mutexes a lot to synchronize threads and lock out whole objects. Therefore I think these "realtime" claims are quite exaggerated.

Re: Baidu File System – A distributed file system for real-time applications

#116
post #27

Earlier quoted context omitted.

I think AFS is still only replicated for read-only, not for read-write.

No, it's replicated read-write, but according to wikipedia, file locks are only machine-wide, so write collisions are easy to create.

Are you sure? From that same wikipedia:

"AFS volumes can be replicated to read-only cloned copies."

Re: Baidu File System – A distributed file system for real-time applications

#117
post #73
post #61

Earlier quoted context omitted.

Sure, I would have preferred a non-branching: std::string pad = descriptive_name_here(path); with the added bonus of being able to add "const" to that, for the benefit of the reader. This is not relying on compiler implementation! Can you name one language that has strings that initialise to anything but a valid object containing an empty string? This is not an obscure side-effect. This is like assuming "std::vector…

In Java the default value for a String (or any object) is null, not an empty string.

Did you not read my whole comment? Please read the whole thing before replying.

Re: Baidu File System – A distributed file system for real-time applications

#118
post #71
post #63

Earlier quoted context omitted.

Except there's already a 'true'. When I see this I know that whoever wrote is completely incompetent (as in "does not know programming", not "is stupid"). It's clever, yes. The bad kind of clever that's also misguided.

Sometimes there is a 'true', sometimes there isn't. Sometimes the code is new, sometimes it is legacy code. You are making too many assumptions. When I see this I know that...

The point of defining true to (1==1) is that it's "future proof" in case implicit typecasting to bool works in a world where 0 evaluates to "true".

That would break approximately ALL C code.

You're being ridiculous. You might as well try to protect against the meaning of "if" changing.

I've seen amateur code that tries to protect against "stdio.h" going away and therefore reimplementing everything in it. This is like that.

Believing that the meaning of everything can change means that you cannot use anything you didn't code yourself. You can't trust documented APIs, then that's some sort of programmer NIH nihilist.

Re: Baidu File System – A distributed file system for real-time applications

#119

Earlier quoted context omitted.

> if your DFS works within a docker container you'll have a very strong differentiator since the rest don't. Untrue. Gluster is already deployed that way in many places. Yes, in production and at scale.

Do you mean hackery of this sort: http://blog.xebia.com/persistence-with-docker-containers-tea... Or do you know of clean, container only (no plugins or special external tools) solution ?

Oh, sorry, didn't realize we were playing the "move the goalposts" game. If you were to google for "gluster" and "containers" you'd get everything from slick marketing stuff to a presentation at the recent Gluster developer summit in Berlin. I have no idea if any of those would meet your next set of standards but, frankly, meh.

Re: Baidu File System – A distributed file system for real-time applications

#120
post #65

Earlier quoted context omitted.

It is specified clearly though. The behavior is not compiler dependent, it's specified in the C++ Language standard. See http://en.cppreference.com/w/cpp/language/default_initializa... and http://www.cplusplus.com/reference/string/string/string/ If a different compiler breaks this behavior, it's not standard compliant and thus could do all sorts of stuff in every possible line, including in: std::string pad = "";

Standards change. Don't they?

So how can you use operator= or copy constructor of string?

The '= ""' won't help you.

Post reply on HN