Live data from Hacker News

Using D to Create the World’s Fastest File System

dlang.org

11–20 of 167 posts

Re: Using D to Create the World’s Fastest File System

#11

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

My experience, which of course is anecdotal, is that the advantage is it's easy to change the algorithms and data structures.

I've maintained C and C++ code bases for decades, and I've found that the first algorithm I tried has stayed there in the code. It gets tweaked, optimized, refactored, but it's the same algorithm and data structure.

With D, when I developed the Warp preprocessor, https://github.com/facebookarchive/warp I continually tried different algorithms & data structures to see which was faster.

One reason it's easier is that in C/C++ the . is used for value field access, while -> for pointer field access. In D, . is used for both. So you can easily switch between a value and a pointer to the value by changing little code.

Re: Using D to Create the World’s Fastest File System

#12
Ah, just looking at this. When I was looking at D back around 2005 it was a closed shop run by one guy - I had the feeling it did not take off because it simply wasn't an open enough project.

It is a shame, as since then we've had go, Swift, Rust all come along and it is a lot more competitive for mindshare in that space... not to forget the merits of modern C++ (it is a far better language post 2011)

Re: Using D to Create the World’s Fastest File System

#13
post #10

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

One of the nice aspects of D is that it interfaces with C++ really nicely. I'm not overly familiar with how Rust handles this, as the last time I tried to glue together Rust and C++ was more than a year ago, and the tooling was in evolution. From Go / CGo, it can be pretty scary to interface with native C++ code, and it requires lots of annoying boilerplate.

"C++ has a very complex ABI, and the Rust ABI is not frozen. However, both C++ and Rust support functions that use the C ABI. Therefore, interoperability between C++ and Rust involves writing things in such a way that C++ sees Rust code as C code and Rust sees C++ code as C code."

https://hsivonen.fi/modern-cpp-in-rust/

Re: Using D to Create the World’s Fastest File System

#15
post #6

Sounds to me like they would be better off with Rust as they are actively avoiding GC. How are the two languages compare in this problem space?

You can avoid GC with a non-GC standard library that they actually wrote: https://github.com/weka-io/mecca

Here's the thing though: They wrote a new compiler and a new standard library to make the language work for them. How exactly is that good publicity for D?

D itself looked and still looks very interesting to me, as it can be very performant, without having insanely ugly syntax such as c++ or rust (inb4 rUsT iS VeRy ReADaBlE).

Re: Using D to Create the World’s Fastest File System

#16

I discovered D is a secret weapon at several companies, but not all of them want to make it widely known...

Several languages seem to share this curse. You do see companies advertise it on language conferences or websites: Jane Street for OCaml, Remedy Entertainment for D-lang, Naughty Dog and ITA for Common Lisp. But you are often left wondering if it is the tip if the iceberg, or an accurate reflection of industry use. In the case of CL, you see multiple commercial implementations still being supported, indicating much more use than one would suspect from grepping github projects.

A pattern I seem to observe is that if the program/framework/platform grows large and successful enough, the company gets acquired for its tech and everything gets rewritten in some more 'industry standard' language.

edit: As pointed out, Jane Street advertises its OCaml use, not Haskell. Oops!

Re: Using D to Create the World’s Fastest File System

#17

I discovered D is a secret weapon at several companies, but not all of them want to make it widely known...

Is this true? Mind speaking on which types of projects it was mostly used?

Remedy (game developer) uses it in-house, mostly for their animation and AI engines.

Re: Using D to Create the World’s Fastest File System

#18

I discovered D is a secret weapon at several companies, but not all of them want to make it widely known...

Several languages seem to share this curse. You do see companies advertise it on language conferences or websites: Jane Street for OCaml, Remedy Entertainment for D-lang, Naughty Dog and ITA for Common Lisp. But you are often left wondering if it is the tip if the iceberg, or an accurate reflection of industry use. In the case of CL, you see multiple commercial implementations still being supported, indicating much m…

I’m nitpicking a bit, but Jane Street focuses on OCaml.

Re: Using D to Create the World’s Fastest File System

#19
post #15

Earlier quoted context omitted.

You can avoid GC with a non-GC standard library that they actually wrote: https://github.com/weka-io/mecca

Here's the thing though: They wrote a new compiler and a new standard library to make the language work for them. How exactly is that good publicity for D? D itself looked and still looks very interesting to me, as it can be very performant, without having insanely ugly syntax such as c++ or rust (inb4 rUsT iS VeRy ReADaBlE).

If D didn't have compelling advantages, they wouldn't have invested such effort.

Also, when one is really after every erg of performance, one often finds a need to work under the hood to improve it for one's special use cases.

For example, many, many, MANY C and C++ users replace malloc/free, and/or write their own custom allocator.

Re: Using D to Create the World’s Fastest File System

#20

I discovered D is a secret weapon at several companies, but not all of them want to make it widely known...

Is this true? Mind speaking on which types of projects it was mostly used?

Congrats for Weka.IO. They managed to pull it through after the CEO visited the DConf and asked for help.

D is a brilliant language. I got drawn into D by this article[1]. I've used D to replace some of the test tools where Java/Python were used. GC is a friend in such cases. I could've written them in C++ as well, but Phobos wins over STL hands down!

Initially the "solution" test application for one of our REST services was written in Java. It covered the functional, performance and longevity tests. The REST service itself was written in C++. Ultimately the customers reported that under heavy load the response time of the REST service is too long (> 400 ms) and we fixed the code as well. But during tests we were unable to really load the application to the fullest. 48 cores & 48 threads in Java and we still couldn't overload the REST service. We had no way to prove that our fix actually worked. Ultimately, we rewrote the test app in D and boom, we were finally able to achieve the load that customers created in their production env - both in terms of response times and throughput. I could've written it in C++. But D's std.parallelism + template mixin is a boon, thanks to David Simcha. Based on our tests, the D test app was 20 times faster than the Java equivalent (written by one of the finest and the expert).

Sure, there is a lot of visual noise due to @nogc, etc, etc. But beneath that there is a beautiful and readable language that's waiting to unfold, if and only if the D authors realize that they stop the feature creep and revert to sane defaults.

On the other hand, I started to convert the REST service from C++ to D last year and it didn't go well. Blame me! There are _very_ rough edges in D as well that are result of over engineering. One such thing is the const'ness in D. It make it impossible to use const, ever, by library authors. I don't care to go and fight in D forum. Manu has already screamed many a times. Lately I found that one of the reasons for poor performance of my D port was also because of auto-decoding! Duh! It was not even GC. (The default should be fast, isn't it?). I spent a considerable amount of time for this port and eventually it failed to meet the performance vs maintainability trade-off. C++ is getting better year by year, at least every 5 years :-). std::vector, std::map, etc are safe for concurrent reads (at least on POSIX). std::vector has erase with iterator. Phobo's std.array doesn't have an equivalent features, albeit it is much more readable than STL.

FWIW, D doesn't deserve the negative publicity that it usually gets. It's a fantastic language. Try it and if you like it, enjoy using it for the right use cases. After all choose the right tool for the job at hand.

[1] https://wiki.dlang.org/Component_programming_with_ranges

Post reply on HN