Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using D to Create the World’s Fastest File System

dlang.org

Post reply on HN
Print view View original

21–30 of 167 posts

Re: Using D to Create the World’s Fastest File System

#21

Post by elemeno »

yvdriess wrote: post #16
dvfjsdhgfv wrote: post #2

I discovered D is a secret weapon at several companies, but not all of them want to make it widely known...

Several languages seem to share this curse. You do see companies advertise it on language conferences or websites: Jane Street for OCaml, Remedy Entertainment for D-lang, Naughty Dog and ITA for Common Lisp. But you are often left wondering if it is the tip if the iceberg, or an accurate reflection of industry use. In the case of CL, you see multiple commercial implementations still being supported, indicating much m…

Just a nitpick, but it's OCaml for Jane Street rather than Haskell.

Haskell is certainly used as a bit of a secret weapon in some of the Quant groups at banks like Barclays Capital though!

Re: Using D to Create the World’s Fastest File System

#22

Post by arunc »

WalterBright wrote: post #11
hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

My experience, which of course is anecdotal, is that the advantage is it's easy to change the algorithms and data structures. I've maintained C and C++ code bases for decades, and I've found that the first algorithm I tried has stayed there in the code. It gets tweaked, optimized, refactored, but it's the same algorithm and data structure. With D, when I developed the Warp preprocessor, https://github.com/facebookarc…

> In D, . is used for both

This is an absolute pleasure to work with. Turtles (.) all the way down (pointers, values, modules, etc)!

Re: Using D to Create the World’s Fastest File System

#23

Post by yvdriess »

hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

D feels much more like a 'sane' C successor, dropped from a parallel dimension where C++ could evolve without caring much about breaking its legacy code running the world. It has a few warts, some D v1 leftovers, but overall feels much more coherent than modern C++; even as the latter has grown much closer to D by piling more on top of its templating system.

What I personally found the stand out feature of D to be, is its compile-time template/macro system. D was designed by a compiler builder, and it shows. Its compile-time meta-programming (generics/templates) was built on compiler reflection, whereas C++ did it by abusing template instantiation. I bet every modern C++ programmer that writes his first "if __traits(compiles, S.s1) ..." gets a huge grin on his face, remembering the C++ SNIFAE abuse needed. A crude analogy would be working with Lisp macros after years of C preprocessor hacking.

In short, I would recommend D for projects where you want better rapid prototyping and feature turn-around than C++, but the end result needs still to be C-style speed.

Re: Using D to Create the World’s Fastest File System

#24

Post by WalterBright »

arunc wrote: post #20
thwy12321 wrote: post #4

Earlier quoted context omitted.

Is this true? Mind speaking on which types of projects it was mostly used?

Congrats for Weka.IO. They managed to pull it through after the CEO visited the DConf and asked for help. D is a brilliant language. I got drawn into D by this article[1]. I've used D to replace some of the test tools where Java/Python were used. GC is a friend in such cases. I could've written them in C++ as well, but Phobos wins over STL hands down! Initially the "solution" test application for one of our REST serv…

The difference between const in D and const in C++ is that D's const is transitive, and C++'s is not.

Transitive const is necessary for things like pure functions. It's more difficult to use because the compiler really, really means it. But when it's there, you (the user) knows that function is not modifying the const data structure, not no-how, not no-way.

Re: Using D to Create the World’s Fastest File System

#25

Post by skocznymroczny »

yvdriess wrote: post #23
hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

D feels much more like a 'sane' C successor, dropped from a parallel dimension where C++ could evolve without caring much about breaking its legacy code running the world. It has a few warts, some D v1 leftovers, but overall feels much more coherent than modern C++; even as the latter has grown much closer to D by piling more on top of its templating system. What I personally found the stand out feature of D to be, i…

I like D, but I'd draw different conclusions. D 1.0 was kind of like a "native C#" for me - a language that combines performance, pointers, direct access and ease of garbage collection, batteries included standard library. I loved it. D 2.0 though, I have more mixed feelings about.

It's trying to be a better C++, but in an effort to woo C++ programmers it's slowly becoming C++ itself. D's curse and blessing is that it tries to cover all bases. Safe programming? Kind of supported. OOP programming? Kind of supported. Functional programming? Kind of supported. GC? Kind of supported. No GC? Kind of supported. Everything is supported to some extent, but it takes some effort to learn what exactly is supported in which subset, what language features don't interact well with each other. It's a big boost for expert D programmers, but for less expert ones it's a lot of mental overhead to worry about.

Re: Using D to Create the World’s Fastest File System

#26

Post by WalterBright »

yvdriess wrote: post #23
hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

D feels much more like a 'sane' C successor, dropped from a parallel dimension where C++ could evolve without caring much about breaking its legacy code running the world. It has a few warts, some D v1 leftovers, but overall feels much more coherent than modern C++; even as the latter has grown much closer to D by piling more on top of its templating system. What I personally found the stand out feature of D to be, i…

If I may be blunt, I could hardly believe SFINAE was a feature, and hated implementing it.

But one thing I liked about C++ was the partial ordering scheme for template selection. I liked it so much D uses it for both templates and function overload selection (as opposed to the multi-level argument matching scheme C++ uses). Once I figured it out, I thought it was a beauty.

Re: Using D to Create the World’s Fastest File System

#27

Post by arunc »

sargun wrote: post #10
hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

One of the nice aspects of D is that it interfaces with C++ really nicely. I'm not overly familiar with how Rust handles this, as the last time I tried to glue together Rust and C++ was more than a year ago, and the tooling was in evolution. From Go / CGo, it can be pretty scary to interface with native C++ code, and it requires lots of annoying boilerplate.

I've written wrappers for C++ libraries to work with D and it's _very_ easy. To work with C++ classes in D, I created factory methods that created/destroyed them with C ABI and used/disposed them respectively.

Working with C is even more easier. There is no FFI, etc. Just use Dstep[1] and get going. Or use dpp[2].

[1] https://github.com/jacob-carlborg/dstep

[2] https://github.com/atilaneves/dpp

Re: Using D to Create the World’s Fastest File System

#28

Post by vthriller »

ileri62 wrote: post #15
StreamBright wrote: post #9

Earlier quoted context omitted.

You can avoid GC with a non-GC standard library that they actually wrote: https://github.com/weka-io/mecca

Here's the thing though: They wrote a new compiler and a new standard library to make the language work for them. How exactly is that good publicity for D? D itself looked and still looks very interesting to me, as it can be very performant, without having insanely ugly syntax such as c++ or rust (inb4 rUsT iS VeRy ReADaBlE).

> D itself looked and still looks very interesting to me, as it can be […] without having insanely ugly syntax such as c++ or rust (inb4 rUsT iS VeRy ReADaBlE).

I did a quick dive into aforementioned stdlib only to stumble upon something I would rather prefer any Rust/C++ version over: https://github.com/weka-io/mecca/blob/f5dc6d9f71983ea7b852ad...

Some might want to exempt standard library from the "readable code" argument, but I'd argue that readability argument was lost the moment language designers allowed anyone to write things like `whatisthis!"whatever"({t.__ctor(args);})` (regardless of best practices that regular devs might unanimously follow). You can't just say that "A is bad 'cause it allows unreadable code" and follow that with "nobody's doing bad things in B therefore it's much better".

Re: Using D to Create the World’s Fastest File System

#29

Post by yvdriess »

ileri62 wrote: post #15
StreamBright wrote: post #9

Earlier quoted context omitted.

You can avoid GC with a non-GC standard library that they actually wrote: https://github.com/weka-io/mecca

Here's the thing though: They wrote a new compiler and a new standard library to make the language work for them. How exactly is that good publicity for D? D itself looked and still looks very interesting to me, as it can be very performant, without having insanely ugly syntax such as c++ or rust (inb4 rUsT iS VeRy ReADaBlE).

THey didn't write a new compiler. They did have some work to make their codebase work on LDC, the LLVM-backend implementation of D.

Rewriting the standard library for performance is not unique to D. Game developers famously avoid stl/stdc++ (https://github.com/electronicarts/EASTL). The reasons behind that seems to be similar: optimizing memory allocation to their specific use case.

Re: Using D to Create the World’s Fastest File System

#30

Post by Mindwipe »

WalterBright wrote: post #11
hardwaresofton wrote: post #8

I would have liked to hear more about what exactly made D so great -- reading through it it's kind of vague. He mentions generics and finally getting LLVM support working as a target, and being able to use D at a high and low-level but I don't see anything here that's groundbreaking. Could someone who uses D Hate to be that guy, but if you could compare it with rust that would be double A++ good... I know the targets…

My experience, which of course is anecdotal, is that the advantage is it's easy to change the algorithms and data structures. I've maintained C and C++ code bases for decades, and I've found that the first algorithm I tried has stayed there in the code. It gets tweaked, optimized, refactored, but it's the same algorithm and data structure. With D, when I developed the Warp preprocessor, https://github.com/facebookarc…

Thanks, this was a good explanation of where the likely benefits originate for the use case in the OP!
Post reply on HN