61–70 of 190 posts
Earlier quoted context omitted.
The problem with C++, for a lot of people, is not that it's impossible to do it right, but that it's easy to do it wrong.
I think this applies to any kind of advanced software development or programming languages, not only C++. There may be many reasons why scientists who are not computer scientists feel more comfortable with Go than with C++, but performance is certainly not one of them.
"We have not performed a detailed comparison against the original version of elPrep implemented in Common Lisp, but based on previous performance benchmarks, the Go implementation seems to perform close to the Common Lisp implementation." LMFAO.
> Most existing Common Lisp implementations use stop-the-world, sequential garbage collectors. To achieve good performance, it was therefore necessary to explicitly control how often and when the garbage collector would run to avoid needless interruptions of the main program, especially during parallel phases. As a consequence, we also had to avoid unnecessary memory allocations, and reuse already allocated memory as far as possible, to reduce the number of garbage collector runs. However, our more recent attempts to add more functionality to elPrep (like optical duplicate marking, base quality score recalibration, and so on) required allocating additional memory for these new steps, and it became an even more complex task and a serious productivity bottleneck to keep memory allocation and garbage collection in check.
Earlier quoted context omitted.
Yeah, further down in the article they track down the gap between cpp and Go/Java to the ref-counting deallocation work. I'm not a cpp expert, but it seems surprising to me that GC would beat ref-counting in any scenario.
I guess I was wrong with this comment: https://news.ycombinator.com/item?id=22959600 1. Reference counting is a form of GC; you could implement a JVM that used reference counting (though in order to be general a small amount of additional work is needed) 2. Reference counting causes extra work every time a reference appears or disappears. Tracing GCs amortize that cost across many allocations. 2.b. This is particular…
It would leak memory - reference counting cannot collect cycles (you need tracing GC for that, defeating the purpose of refcounting).
I had a quick look at the C++ source code provided at https://github.com/ExaScience/elprep-bench/tree/master/cpp . As suspected, everything is dynamically allocated and no memory mapping (see e.g. http://man7.org/linux/man-pages/man2/mmap.2.html ) is used. No wonder this is slow and eats a lot of memory. At the moment I have no information about why this design was chosen, if there is a justification for it, or if th…
> elPrep is an open-ended software framework that allows for arbitrary combinations of different functional steps in a pipeline, like duplicate marking, sorting reads, replacing read groups, and so on; additionally, elPrep also accommodates functional steps provided by third-party tool writers. This openness makes it difficult to precisely determine the lifetime of allocated objects during a program run > Phase 1 all…
Earlier quoted context omitted.
I think this applies to any kind of advanced software development or programming languages, not only C++. There may be many reasons why scientists who are not computer scientists feel more comfortable with Go than with C++, but performance is certainly not one of them.
The authors of that paper are computer scientists.
No Rust smh
It was mentioned in the article and was not considered a candidate due to a specific API requirement: > Other mature programming languages with support for reference counting include Objective-C, Swift, and Rust [50]. However, in its algorithm for duplicate marking, elPrep requires an atomic compare-and-swap operation on reference-counted pointers, which does not exist in those languages, but exists in C++17.
I had a quick look at the C++ source code provided at https://github.com/ExaScience/elprep-bench/tree/master/cpp . As suspected, everything is dynamically allocated and no memory mapping (see e.g. http://man7.org/linux/man-pages/man2/mmap.2.html ) is used. No wonder this is slow and eats a lot of memory. At the moment I have no information about why this design was chosen, if there is a justification for it, or if th…
> elPrep is an open-ended software framework that allows for arbitrary combinations of different functional steps in a pipeline, like duplicate marking, sorting reads, replacing read groups, and so on; additionally, elPrep also accommodates functional steps provided by third-party tool writers. This openness makes it difficult to precisely determine the lifetime of allocated objects during a program run > Phase 1 all…
Or like this:
auto alns = make_shared>>();
It's like make_shared is the only way they know to allocate an object. They even put string inside of shared_ptr:
class istream_wrapper {
...
shared_ptr buffer;
EDIT: And they don't even seem to use std::move anywhere :/ There's a huge gap between this code and custom allocator territory.
Like what is this? https://github.com/ExaScience/elprep-bench/blob/master/cpp/f...
auto alns = any_cast>>>(data);
try { any_cast(data) ; }
catch ( bad_any_cast ) { throw runtime_error(...) ; }
They really like std::deque and use it everywhere even though the sizeof(T) is like a few dozen bytes at best so they should rather use std::vector. The data structure of deque is a list of array. while it can amortize the continious adding of the elements to front or back, since the element size is very small, they should rather use vector.
Speaking of data structure, they also use std::unordered_map. the unordered_map is very slow(it's a node based hash map, not suitable for the modern hardware) and the sizeof(int) + sizeof(any) is like 20 bytes(sizeof(int) + two pointers) so they got no benefit of using node based data structure here. They should rather use sorted vector and binary search it.
My conclusion, it's slow because they wrote C++ like a dynamic typed language and they choose the wrong data structures.
This is not surprising. Parallel GC will almost always be faster than refcounting. If you wrote a C or C++ program to accomplish this task after carefully planning out exactly when stuff needs to get manually malloced/freed, you could outperform any of their approaches And like another commenter mentioned, if you're writing a program which streams a lot of data sequentially from disk, and where throughput is importan…
If you are seeking randomly and doing small reads, then mmap will help quite a bit: the data will be faulted in, and accessing it a second, third, or hundredth time will not cost much.