Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster Command Line Tools in D

dlang.org

Post reply on HN
Print view View original

71–80 of 100 posts

Re: Faster Command Line Tools in D

#71

Post by bionsuba »

ajbonkoski wrote: post #62

I don't get articles like this.. they seem to miss the bigger point. Typically there are two modes in my computing: (1) Scripting / Command-line get stuff done and throw-away, and (2) Serious applications that are heavily used, need to process lots of data and be as fast as possible (e.g. processing millions of files like this one where the algorithm constant factor really matters). In case 1: Hack something together…

This is a nice sentiment, but this rarely plays out in practice in my experience.

People use Python all the time to manipulate data sets >= 100G in size despite its speed failings at that size. Why? Because Pandas is just so damn convenient. It would take me a grand total of 30 seconds to write Pandas code which read a TSV and gave me the sum of two multiplied together columns grouped by the day of a timestamp column. Doing that in C would take several orders of magnitude more time.

It's an optimization of people's time problem. You could probably spend several hours (or days) writing a C program for a specific problem. But if you can spend only 40% of the time writing the program and have it only 20% slower, then that's a definite win (these numbers are just an example).

Re: Faster Command Line Tools in D

#72

Post by acehreli »

gregwebs wrote: post #67

If you are interested in fast TSV tooling you might also try xsv [1], which is written in Rust and has similar features to the linked tsv-utils-dlang project. The frequency command computes the frequency of values in 2 columns in xsv frequency -n -s 1,2 -l 100 googlebooks-eng-all-1gram-20120701-0.tsv 5.19s user 0.06s system 351% cpu 1.490 total [1] https://github.com/BurntSushi/xsv

xsv appeares in comparisons on the author's tsv-utils-dlang repo: https://github.com/eBay/tsv-utils-dlang/blob/master/docs/Per...

Re: Faster Command Line Tools in D

#73

Post by acehreli »

ajbonkoski wrote: post #62

I don't get articles like this.. they seem to miss the bigger point. Typically there are two modes in my computing: (1) Scripting / Command-line get stuff done and throw-away, and (2) Serious applications that are heavily used, need to process lots of data and be as fast as possible (e.g. processing millions of files like this one where the algorithm constant factor really matters). In case 1: Hack something together…

It's one of the points of this article that you don't need those two modes with D. Scripting and custom code in the same language is possible but it's still an alien concept.

Re: Faster Command Line Tools in D

#74

Post by pixel_fcker »

destructionator wrote: post #63
milcron wrote: post #43

Earlier quoted context omitted.

D's standard library uses garbage collection.

So do Java, C#, Python, Ruby, PHP, Javascript, and virtually everything else and they are very heavily used. Garbage collection is a smashing success in the real world and D made the right decision to follow that success. Of course, it is also true that much of the standard library doesn't actually use it... but these objections are never actually about facts.

None of those languages you listed are for systems programming, which is the one niche you need to excel at if you want to replace C and C++. People who care about that stuff tend to care a lot about managing memory.

Re: Faster Command Line Tools in D

#75

Post by throwaway7645 »

gshulegaard wrote: post #51
throwaway7645 wrote: post #50

Earlier quoted context omitted.

No, it isn't debatable which is generally faster (D), but I was impressed that you could just prototype in Python (probably significantly faster than D) and run it through PyPy when you're done and get 90% of the performance of D. Of course Python has the bloat of the interpreter and Jitter, while D is just a binary. Point is I was expecting more speed from D. I'm curious if this is just luck or the two would be neck…

Optimizing/squeezing performance out of Python is a rabbit hole: https://www.ibm.com/developerworks/community/blogs/jfp/entry... I would speculate using numba or Cython would yield further performance gains over PyPy...but that's mostly just based on anecdotal comparisons: https://cardinalpeak.com/blog/faster-python-with-cython-and-... I just think it is a bit dishonest to try and make a claim as pointed as this arti…

I think they're just giving you bounds on what to expect and not selling anything. The D optimizations looked a lot easier (write it slightly different) than mucking with Cython or Numba. Simply running through PyPy is another thing all together.

Re: Faster Command Line Tools in D

#76

Post by gorthol »

lenkite wrote: post #28
penpapersw wrote: post #12

Earlier quoted context omitted.

Honest question: what strengths does Go have over D? I became very proficient at Go several years ago, and was reading about D for hours and hours last night, and it looks like D is overall a much better language.

Golang: Easy concurrency using go-routines, a (comparatively) good minimal latency GC, an excellent comprehensive and stable standard library and few bugs. Dlang: Traditional concurrency (thread based), great generics implementation, nice algorithm/container libraries and C++ interop

Actually, D suports "fibers" where you have a "Task". Then you can switch the task handler to another implementation and have Go-like M:N coroutines. The default handlers in the stdlib are only thread based or single-thread coroutines but Vibe.d for example supports Go-like coroutines.

Still, Go is really hard to beat on that front because it makes it super easy and has the really brilliant feature of making almost everything that waits for IO interruptable inside a function when you call "go function()" without having to mark operations with "async".

Re: Faster Command Line Tools in D

#77

Post by xtreak_1 »

lerax wrote: post #32
maxpert wrote: post #30

Makes me wonder, how many people actually use D-lang in production. Specifically with HTTP stack what kind of numbers/performance benchmarks we are looking at? Other than toy projects is there a company out there running D on massive scale (millions per day)?

Netflix and Machine Learning with D.

There was a similar comment : https://news.ycombinator.com/item?id=14064012 who also seems to work on Machine Learning with D at Netflix. It will be a great help if you can add it to https://dlang.org/orgs-using-d.html or share the experience and scale at which its being used at Netflix if its not a problem . Thanks .

Re: Faster Command Line Tools in D

#78

Post by gorthol »

aisofteng wrote: post #49
mhh__ wrote: post #21

Earlier quoted context omitted.

I can only give my own biased opinion: Go's only advantage is it's commercial adoption e.g. builtin in support in AppEngine and co. I am not referring to it's ecosystem: While gofix and friends are very good, D has similar tools available + the usual bells and whistles for IDEs. By chance, I found (while looking for a rob pike quote) this article critiquing Go's design: It uses D to demonstrate it's arguments. http:/…

The first code comparison in that article compares Go error handling with doing a `catch Exception` in D. I don't find that a fair comparison; `catch Exception` for a block of code that can fail in different ways at different points is a bad practice.

It depends. If you don't care about the specific error and just want to notify the failure up the stack or recovering is perfectly fine. A lot of Go code with the same intent just do a lot of "if err!=nil return err" which is the same but writing for every function call.

Re: Faster Command Line Tools in D

#79

Post by gorthol »

yumaikas wrote: post #46
penpapersw wrote: post #12

Earlier quoted context omitted.

Honest question: what strengths does Go have over D? I became very proficient at Go several years ago, and was reading about D for hours and hours last night, and it looks like D is overall a much better language.

The other strength that Go has over D is that Go, being used by Googlers, is much more likely to have better networking support. Two examples come to mind: HTTP2 support (Go was used for either the first or second implementation of that and has standard library support), and TLS cryptography, where Go has it's own suite written by experts, where a good portion of the non Microsoft ecosystem relies on OpenSSL. For web…

There is Vibe.d but I don't know if is as good as Go stdlib.

Re: Faster Command Line Tools in D

#80

Post by gorthol »

cwyers wrote: post #44
maxxxxx wrote: post #38

Earlier quoted context omitted.

D would make a good case study to look at why some technologies find widespread use whereas others don't. I don't understand why it never took over from C++.

C++ was deliberately backwards compatible with C. D was not backwards compatible with C or C++.

D uses C calling conventions and struct layout so it's pretty trivial to call D code from D or vicecersa and link together, the only step added over C++ is writing an include file with the declarations of the stuff you want to call.
Post reply on HN