Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of C++17, Go, and Java for a next-generation sequencing tool

bmcbioinformatics.biomedcentral.com

Post reply on HN
Print view View original

161–170 of 190 posts

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#161

Post by benibela »

gpderetta wrote: post #83
benibela wrote: post #80

Earlier quoted context omitted.

>the unordered_map is very slow(it's a node based hash map, not suitable for the modern hardware) It is very slow? I wrote my app in FreePascal. I need a hashmap, but FreePascal has no real standard hashmap, so I have been benchmarking Pascal hashmaps for weeks/months. Today I added std::unordered_map for comparison. It will still take a day to run the benchmark, but so far it looks that std::unordered_map is 25% fas…

Depends what you do. If you populate it only once and then you do a bunch of lookups, assuming a low load factor, you'll be paying for a modulo operation and an extra indirection, not great but not terrible either. If you do a lot of inserts and removal or your load factor is high, then the performance is going to be not great.

My benchmark is one insert, multiple lookups, repeat.

I have no use for deletions. Although when I want to make a general benchmark, I probably should measure it once, too

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#162

Post by gpderetta »

the8472 wrote: post #160
gpderetta wrote: post #108

Earlier quoted context omitted.

That's 2CAS. A true DCAS works on two arbitrary sized locations, cmpxchg16b acts on two contiguous pointer sized locations.

I am aware, but that restriction shouldn't affect its use for an atomic ref-counted pointer since you can place the count and the pointer next to each other.

How would it work exactly? The layout of a generic shared ptr (but not specifically std::shared_ptr) is:

   ptr -> (count, payload)
instead of:

   (count, ptr) -> payload
As you can see, count is not alongside the pointer itself as distinct instances of ptrs need to share the count.

Let's say you want to acquire an additional reference to ptr. You need to both copy the current value of ptr and increment count atomically to protect against concurrent modifications of ptr that might drop the last reference to payload.

Delaying the deallocation of payload via hazard pointers, RCU or some other deferred reclamation scheme works, but it is significantly more complex. I believe this is what the rust arc-swap package does internally.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#163

Post by [deleted] »

movedx wrote: post #121
ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

> My conclusion, it's slow because they wrote C++ like a dynamic typed language Here's my conclusion without needing to look at the code: is C++17 so complicated a language that having to understand deeper concepts such as the ones you've mentioned means a reduced "time to market" for marginal gains over the same code written in Go? Or put another way: did you just further prove that Go is the better option because i…

[deleted]

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#164

Post by imtringued »

WesternStar wrote: post #105
cozzyd wrote: post #89

Earlier quoted context omitted.

why not just define std::dict or something like that that is better?

Because you should be able to find a good hash data structure for your language. You shouldn't be able to accidentally get a bad one. It should arguably be the first data structure you reach for.

Deprecate the old data structure. Even Java did that. Originally it had the Vector class as primary List type. Nowadays everyone uses the List interface and ArrayLists as default implementation.

You can even find this in the javadoc of the Vector class:

>As of the Java 2 platform v1.2, this class was retrofitted to implement the List interface, making it a member of the Java Collections Framework. Unlike the new collection implementations, Vector is synchronized. If a thread-safe implementation is not needed, it is recommended to use ArrayList in place of Vector.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#165

Post by the8472 »

gpderetta wrote: post #162
the8472 wrote: post #160

Earlier quoted context omitted.

I am aware, but that restriction shouldn't affect its use for an atomic ref-counted pointer since you can place the count and the pointer next to each other.

How would it work exactly? The layout of a generic shared ptr (but not specifically std::shared_ptr) is: ptr -> (count, payload) instead of: (count, ptr) -> payload As you can see, count is not alongside the pointer itself as distinct instances of ptrs need to share the count. Let's say you want to acquire an additional reference to ptr. You need to both copy the current value of ptr and increment count atomically to…

Ah you are right, I was only considering the limited case of replacing the value when the reference count is 1.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#166

Post by imtringued »

zmmmmm wrote: post #107

It has to be noted: it's quite a strange approach all round - this framework reads all the data into memory. So if you have a 100GB genome it will read 100GB into memory. Presumably it stays in memory uncompressed so we are talking hundreds of GB to process even a single whole genome sample. This may indeed have some performance benefits, but it's a very impractical approach from a hardware point of view. Few places…

We once asked our cluster department to give us bare metal access to their smallest machine. They gave us a dedicated server with 256GB RAM. The real cluster nodes are bigger than that because they handle multiple jobs at the same time and they have hundreds of them.

This isn't some electron framework where it is unexcusable to use that much RAM. The hardware that is available to scientists is more than capable of handling these workloads.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#167

Post by imtringued »

voldacar wrote: post #46

This is not surprising. Parallel GC will almost always be faster than refcounting. If you wrote a C or C++ program to accomplish this task after carefully planning out exactly when stuff needs to get manually malloced/freed, you could outperform any of their approaches And like another commenter mentioned, if you're writing a program which streams a lot of data sequentially from disk, and where throughput is importan…

It doesn't require care. By default you just put stuff on the stack in C++. You have to go out of your way to use things like ref counting.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#168

Post by grovellogic »

IshKebab wrote: post #150
ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

Maybe they wrote it really badly so someone will take offence and rewrite it for them! Also in my experience `std::deque` is almost always slower than `std::vector`. In theory it shouldn't be, but copying contiguous data is so fast on modern CPUs that the reallocation of `std::vector` (or occasional front-insertion) is very cheap.

In theory copying contiguous data is faster than dealing with pointer indirection.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#169

Post by adev_ »

movedx wrote: post #129
kllrnohj wrote: post #124

Earlier quoted context omitted.

If you know Go and don't know C++ then yes absolutely use Go. If you're trying to do a performance comparison though it doesn't seem unreasonable to have "basic" familiarity with the languages being compared? None of the knowledge mentioned is "esoteric". We're not talking hyper optimized insane C++ optimizations here with custom allocators and specialized code generation with templates. We're talking "basic" memory…

> If you're trying to do a performance comparison though it doesn't seem unreasonable to have "basic" familiarity with the languages being compared? No. Because it's not just a performance comparison, really. Without realising it, it's also a learning test. The very fact they wrote sub-par C++ code, likely without realising it, and then proceeded to produce (sub-par?) Go code that was faster demonstrates the difficul…

> No. Because it's not just a performance comparison, really. Without realising it, it's also a learning test. The very fact they wrote sub-par C++ code, likely without realising it, and then proceeded to produce (sub-par?

Who or what tell you it's a learning test ? It is a pretty weak asumption.

The publication says nothing on the actual past experience of the developers in every language. They might very well be experienced in Go or Java.

From their C++, It is pretty obvious it is not their main language.

In my experience, this kind of code of nested shared_ptr everywhere is pretty typical of developers with a Java background just starting C++.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#170

Post by kazinator »

Rochus wrote: post #50
aidenn0 wrote: post #35

Earlier quoted context omitted.

> > Common Lisp also lets you use mmap > But not without allocating dynamic memory and copying data. Sure it does. In SBCL you can force a stack allocation (though rarely does it improve performance), and very short-lived values do not leave registers in any case. > > They clearly wanted automatic memory management > Most likely because of some misconceptions. There are both good and bad reasons to want automatic mem…

Common Lisp: It is permissible for an implementation to simply ignore such declarations. And you still have to copy. Ref counting: only makes sense in a few special cases. Avoiding dynamic memory management: have a look at mmap.

It's permissible for a C compiler to emit shell scripts.
Post reply on HN