Comparison of C++17, Go, and Java for a next-generation sequencing tool • Hacker News Board

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#71

Post by jayd16 » Thu, Apr 23, 2020, 10:03 PM UTC

etse wrote: post #44

DeathArrow wrote: post #30

Earlier quoted context omitted.

True. C# and Java constantly beat Swift on speed.

Sorry about the naive question, but if the memory management overhead is worse in Swift, is the hardware it runs on typically better? I'm assuming some of this because I've noticed Android devices tend to require more CPU/memory compared to iOS devices in the same generation.

There are a lot of factors at play. Some technical like OS differences, and some non-technical like race to bottom spec pushing by Android OEMs.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#72

Post by WesternStar » Thu, Apr 23, 2020, 10:08 PM UTC

ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

We should break ABI on unordered map just to stop embarrassing ourselves in public and in front of new users.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#73

Post by gpderetta » Thu, Apr 23, 2020, 10:11 PM UTC

ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

The use of 'any' and the ridiculous levels of boxing is bizarre.

There is also absolutely no attempt at using std::move; the amount of gratuitous copies and refcount updates is staggering.

Having said that, apart from the use of std::any, the code is fairly clean even though obviously is not even remotely written for performance..

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#74

Post by gpderetta » Thu, Apr 23, 2020, 10:14 PM UTC

voldacar wrote: post #46

This is not surprising. Parallel GC will almost always be faster than refcounting. If you wrote a C or C++ program to accomplish this task after carefully planning out exactly when stuff needs to get manually malloced/freed, you could outperform any of their approaches And like another commenter mentioned, if you're writing a program which streams a lot of data sequentially from disk, and where throughput is importan…

well, there is careful planning and there is going out of the way to heap allocate everything.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#75

Post by gpderetta » Thu, Apr 23, 2020, 10:16 PM UTC

ori_b wrote: post #70

voldacar wrote: post #46

This is not surprising. Parallel GC will almost always be faster than refcounting. If you wrote a C or C++ program to accomplish this task after carefully planning out exactly when stuff needs to get manually malloced/freed, you could outperform any of their approaches And like another commenter mentioned, if you're writing a program which streams a lot of data sequentially from disk, and where throughput is importan…

That's a case where mmap isn't actually all that much faster than read, and due to the inter processor interrupts needed to synchronize the memory mappings across cores, it may end up much slower. You're grabbing large chunks and flushing the TLB a whole lot. If you are seeking randomly and doing small reads, then mmap will help quite a bit: the data will be faulted in, and accessing it a second, third, or hundredth…

read mapping are actually not too bad. remote TLBs can be sychronised lazily on a page fault.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#76

Post by RantyDave » Thu, Apr 23, 2020, 10:22 PM UTC

WesternStar wrote: post #72

ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

We should break ABI on unordered map just to stop embarrassing ourselves in public and in front of new users.

What's wrong with unordered map?

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#77

Post by zozbot234 » Thu, Apr 23, 2020, 10:30 PM UTC

lpghatguy wrote: post #53

shrimp_emoji wrote: post #42

No Rust smh

It was mentioned in the article and was not considered a candidate due to a specific API requirement: > Other mature programming languages with support for reference counting include Objective-C, Swift, and Rust [50]. However, in its algorithm for duplicate marking, elPrep requires an atomic compare-and-swap operation on reference-counted pointers, which does not exist in those languages, but exists in C++17.

Except that this "atomic" operation over shared_ptr's in the C++ standard isn't actually required to be lock free, and is in fact not lock free in common stdlib implementations. So they're not actually gaining anything over, e.g. RwLock>> in Rust.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#78

Post by ezoe » Thu, Apr 23, 2020, 10:32 PM UTC

WesternStar wrote: post #72

ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

We should break ABI on unordered map just to stop embarrassing ourselves in public and in front of new users.

The problem is, hash map isn't that effective in modern hardware, the overhead cost is greater than the benefit. If it's consists of clustered network connected computers, it may be, but not for the single local computer for the most of problem.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#79

Post by ezoe » Thu, Apr 23, 2020, 10:37 PM UTC

RantyDave wrote: post #76

WesternStar wrote: post #72

Earlier quoted context omitted.

We should break ABI on unordered map just to stop embarrassing ourselves in public and in front of new users.

What's wrong with unordered map?

It's implementation is buckets divided node based hash map. Theoretically, its order is good, but not that efficient in modern hardware where the memory access is heavily cached so the data locality is more important than saving the memory copy or memory size. Using the linked list to save some memory copy doesn't benefit at all and the overhead is far greater than simply copying the contagious large chunk of memory. Especially so if the data type is trivial so the simple byte by byte memcpy is suffice.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#80

Post by benibela » Thu, Apr 23, 2020, 10:40 PM UTC

ezoe wrote: post #69

After quickly glancing the code, I concluded that they wrote C++ like there is no static type. It seems they faithfully ported the very dynamic nature of their existing code to C++ without thinking. Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f... auto alns = any_cast >>>(data); So the data is sam_alignment type inside shared_ptr inside deque inside another share_ptr inside god forbi…

>the unordered_map is very slow(it's a node based hash map, not suitable for the modern hardware)

It is very slow?

I wrote my app in FreePascal. I need a hashmap, but FreePascal has no real standard hashmap, so I have been benchmarking Pascal hashmaps for weeks/months.

Today I added std::unordered_map for comparison. It will still take a day to run the benchmark, but so far it looks that std::unordered_map is 25% faster than the fastest map in the FreePascal standard library. And the best map of 45 Pascal maps is only 30% faster than std::unordered_map. Only 10 maps are faster, and 35 maps are slower.