Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of C++17, Go, and Java for a next-generation sequencing tool

bmcbioinformatics.biomedcentral.com

Post reply on HN
Print view View original

151–160 of 190 posts

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#151

Post by yvdriess »

zmmmmm wrote: post #107

It has to be noted: it's quite a strange approach all round - this framework reads all the data into memory. So if you have a 100GB genome it will read 100GB into memory. Presumably it stays in memory uncompressed so we are talking hundreds of GB to process even a single whole genome sample. This may indeed have some performance benefits, but it's a very impractical approach from a hardware point of view. Few places…

Untrue. Large memory nodes are par the course for genomics workloads. Only a few stages of the analysis pipeline can stream the in/out effectively. Even then, going to disk for the result only to be read back is going to bottleneck your pipeline.

The authors go in more detail around this topic in: https://journals.plos.org/plosone/article?id=10.1371/journal...

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#152

Post by Rochus »

aidenn0 wrote: post #118
Rochus wrote: post #50

Earlier quoted context omitted.

Common Lisp: It is permissible for an implementation to simply ignore such declarations. And you still have to copy. Ref counting: only makes sense in a few special cases. Avoiding dynamic memory management: have a look at mmap.

C++: It is permissible for an implementation to allocate every local variable on the heap. Going back to my original point, you are suggesting a complete rearchitecture of their allocation system. That does not require switching languages to C++. If we are talking about working with 100s of GB of data, that's probably even the correct approach! TFA does not, however, claim that they have a working set of 100s of GB o…

So we are glad that we can use the data of the memory mapped file directly and do not have to allocate or copy anything. But of course you may solve the problem badly if you prefer so; after all, there are even "scientific" publications that do it that way, as the example shows.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#153

Post by yvdriess »

voldacar wrote: post #46

This is not surprising. Parallel GC will almost always be faster than refcounting. If you wrote a C or C++ program to accomplish this task after carefully planning out exactly when stuff needs to get manually malloced/freed, you could outperform any of their approaches And like another commenter mentioned, if you're writing a program which streams a lot of data sequentially from disk, and where throughput is importan…

That makes the assumption that any refcounted object lifetime management can be replaced with manual management, which is not the case.

As the paper states: it is not possible to manually determine the object lifetimes due to the nature of the problem, data and openness of the configurable analysis pipeline. Manual alloc/free would in their case mean holding on to much more memory than a GC/refcounting solution would.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#154

Post by tomp »

aidenn0 wrote: post #114
tomp wrote: post #63

Earlier quoted context omitted.

> you could implement a JVM that used reference counting It would leak memory - reference counting cannot collect cycles (you need tracing GC for that, defeating the purpose of refcounting).

From right after what you quoted: > (though in order to be general a small amount of additional work is needed) Not also that there are two methods of cycle detection for a reference counted GC that are not just a backup tracing-GC 1. Trial deletion (known since at least the mid 80s) 2. Various tracing systems that exploit extra information known to reference-counted systems e.g. Levanoni/Petrank[1] which actually im…

Thanks, I didn't know that!

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#155

Post by magicalhippo »

movedx wrote: post #128
comex wrote: post #125

Earlier quoted context omitted.

‘Esoteric’ is in the eye of the beholder. Even in Go you gain performance by avoiding boxing. > If I can get the same work done in Go and I can learn Go in a week versus months for C++17 (which is far more complex), why would I pick C++17? Because C++17 code if written well is likely to be much faster. If you don’t care about that, by all means stick with Go. But keep in mind that learning is a one-time cost, and you…

> Even in Go you gain performance by avoiding boxing. Go has boxing? > Because C++17 code if written well is likely to be much faster. I was hoping someone could prove this, actually. I don't have the means or the knowledge.

> I was hoping someone could prove this, actually.

Maybe this presentation by Jason Turner from CppCon 2016 can illustrate just what can be done in C++17. His target in the presentation is a C64, but I think it illustrates well just what you can do with the language, and hence what tools are available for writing fast code.

https://www.youtube.com/watch?v=zBkNBP00wJE

edit: If you just want a quick fix, check out what a simple keyword does here: https://youtu.be/zBkNBP00wJE?t=1616

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#156

Post by pjmlp »

59nadir wrote: post #146
pansa2 wrote: post #137

Earlier quoted context omitted.

For a language whose purpose is high-performance, the std:: containers are an embarrassment. `deque` is slow, `map` is slow - so the language added `unordered_map` which is also slow... Are any of the containers (other than `vector`) worth using in high-performance code? I’m sympathetic to the game developers who I’ve heard say “we use C++, but nothing from std::”.

STL is mostly garbage, which is a big reason it's not used in places that actually need performance, both for build times and runtime. Yes, it's embarrassing. I think it should be noted that the performance mantra is mostly for show with C++, though. The same people who spout it will also blatantly use pessimizing language features just because, when normal, procedural code would've done the job faster and ultimately…

About 1% of developers actually need to deliver µs fast code.

The remaining 99% are happy to be able to deliver fast enough portable code without having to reinvent data structures all the time.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#157

Post by benibela »

FpUser wrote: post #112
benibela wrote: post #80

Earlier quoted context omitted.

>the unordered_map is very slow(it's a node based hash map, not suitable for the modern hardware) It is very slow? I wrote my app in FreePascal. I need a hashmap, but FreePascal has no real standard hashmap, so I have been benchmarking Pascal hashmaps for weeks/months. Today I added std::unordered_map for comparison. It will still take a day to run the benchmark, but so far it looks that std::unordered_map is 25% fas…

Free Pascal actually does have hashmap. Check this for reference: https://wiki.freepascal.org/Data_Structures,_Containers,_Col...

But that lists like ten classes which map/hash in their name.

None of them stands out, as being THE standard Pascal map to use for everything.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#158

Post by tom_mellior »

04091948 wrote: post #104
tom_mellior wrote: post #96

Earlier quoted context omitted.

> They really like std::deque and use it everywhere even though the sizeof(T) is like a few dozen bytes at best so they should rather use std::vector. The data structure of deque is a list of array. while it can amortize the continious adding of the elements to front or back, since the element size is very small, they should rather use vector. Are you saying that access patterns (push/pop front/back) never matter and…

in CppCon 2016: Chandler Carruth “High Performance Code 201: Hybrid Data Structures"[1] he says: "but the standard deque data type is really quite bad. I wouldn't recommend anyone use it for anything. if you look at the implementation constraints, the constraints placed upon its iterators, and its invalidation constraints, it's painted into a very unpleasant corner and it has very few opportunities to be an efficient…

Thanks for the reference. But this is still specific to access patterns: You can use a deque as a queue without iterating, so iterators and their invalidation constraints should not matter.

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#159

Post by gpderetta »

beached_whale wrote: post #111
gpderetta wrote: post #73

Earlier quoted context omitted.

The use of 'any' and the ridiculous levels of boxing is bizarre. There is also absolutely no attempt at using std::move; the amount of gratuitous copies and refcount updates is staggering. Having said that, apart from the use of std::any, the code is fairly clean even though obviously is not even remotely written for performance..

A copy in function arguments often is not expressed by the compiler and often allows more optimizations.

For relatively small objects with no internall allocation, definitely yes.

Otherwise only a qualified yes: if you are copying it anyway in the body, pass by value allow moving object into and actually avoid copies, but the code linked doesn't use move at all anywhere.

As they seem fairly versed in C++, the only reason for doing that is that they want their code to be modificable by non-programmers, so they want to use a style that is safe and avoids possible use after move.

For this scenario, then yes, C++ is going to be slower than a language with a proper garbage collector (although I would like to see a version with a proper shared_string implementation instead of the abomination that is shared_ptr).

Re: Comparison of C++17, Go, and Java for a next-generation sequencing tool

#160

Post by the8472 »

gpderetta wrote: post #108
the8472 wrote: post #98

Earlier quoted context omitted.

> without a true DCAS (which pretty much no architecture implements) Does x86-64's cmpxchg16b not qualify?

That's 2CAS. A true DCAS works on two arbitrary sized locations, cmpxchg16b acts on two contiguous pointer sized locations.

I am aware, but that restriction shouldn't affect its use for an atomic ref-counted pointer since you can place the count and the pointer next to each other.
Post reply on HN