Live data from Hacker News

Writing a custom iterator in modern C++

internalpointers.com

31–40 of 79 posts

Re: Writing a custom iterator in modern C++

#31

It would be nice to write a DSL to generate some of the C++ boilerplate (say, containers, iterators, etc) required as a baseline before starting to solve the actual problem. A DSL could codify rules like the ones described in this article and generate at least a rough first pass to be hand-tuned later.

Wow, no, really? wouldn't hat would spoil the "fun" of all the endless arcana?

Re: Writing a custom iterator in modern C++

#32

Somewhat related but C++20 ranges do not replace iterators and instead build a higher level of abstraction upon iterators. Ranges are implemented in terms of iterator pairs.

Yeah. That actually sucks; often iterators doesn't fit the iteration model of some sequences, but ranges would fit right in if they didn't require separability into two iterators.

Re: Writing a custom iterator in modern C++

#33
post #9

There's six different kinds of iterator?! Off hand, is it just me or... - Isn't an iterator that outputs data to its recipient container not an iterator, but rather some type of filling API? - Isn't a random access iterator not an iterator but rather a slice view? - Isn't a contiguous iterator a leaky abstraction in that an iterator is just an interface that shouldn't expose how the data backing it is stored? This it…

It's funny how two of the most common bandwagon posts you can see on HN and other programming venues is: * C++ is a terrible bloated language that no one should use. * The power of modern computers is wasted on lazy software engineers writing software with too many layers of abstraction.

I mean, C++ has lots of compile-time abstractions, but unlike with other languages, it can collapse a lot of them during compilation via its inlining and other optimizations. You don't get that with a language like Python...

Re: Writing a custom iterator in modern C++

#34
post #26

Earlier quoted context omitted.

Iterators are inconsistent, but this plays to the strengths of C++'s duck-typed generics. The same code can either copy or insert, depending on whether you use std::begin or std::back_inserter; this isn't principled but it's powerful. The "six types of iterators" overstates it. You don't need to sweat those details unless you are writing boost or something. Just add what you need. I miss C++ iterators in other langua…

Backing up a utf8 stream is nontrivial though.

This is an interesting point. Walking backwards through a UTF8 byte stream is easy: just look for the next non-continuation byte, and Rust really can do this, though the API is buried under `str.chars().rev()`.

C++ is structured around iterators that point into; this enables e.g. writing into the guts of a std::map but not the guts of a std::string if you care about Unicode, since the write might change the string's byte length.

Re: Writing a custom iterator in modern C++

#35
post #9

There's six different kinds of iterator?! Off hand, is it just me or... - Isn't an iterator that outputs data to its recipient container not an iterator, but rather some type of filling API? - Isn't a random access iterator not an iterator but rather a slice view? - Isn't a contiguous iterator a leaky abstraction in that an iterator is just an interface that shouldn't expose how the data backing it is stored? This it…

It's funny how two of the most common bandwagon posts you can see on HN and other programming venues is: * C++ is a terrible bloated language that no one should use. * The power of modern computers is wasted on lazy software engineers writing software with too many layers of abstraction.

I don't see a contradiction between these two statements. There are more languages than C++ and Python. You could use Rust, or some of the languages targeting the JVM or the CLI.

Re: Writing a custom iterator in modern C++

#36

Somewhat related but C++20 ranges do not replace iterators and instead build a higher level of abstraction upon iterators. Ranges are implemented in terms of iterator pairs.

Yeah. That actually sucks; often iterators doesn't fit the iteration model of some sequences, but ranges would fit right in if they didn't require separability into two iterators.

Can you give an example of such a sequence?

Re: Writing a custom iterator in modern C++

#37

Earlier quoted context omitted.

Random access iterators mean you can efficiently advance or retreat an iterator by an arbitrary amount. So a vector (via pointer arithmetic) but not a linked list.

Right but why should this matter at the type level? All you need to implement is next() and annotate it with documentation of the performance characteristics.

Yeah, this is a good question. Scott Myers has a talk where he pokes fun at the "guaranteed O(N log N)" sort of a linked list, where the guarantee is in number of comparisons and not the (very large) number of pointer chases.

Why describe at the type level the difference between a backwards pointer chase and a pointer decrement? The hope is: if you know at compile time that your iterator is efficient in this way, you can pick a more specialized algorithm, say, quicksort instead of mergesort. C++ really does enable this at the type level through template specialization. In practice the STL collections haven't lived up to this, and I think the iterator tags are mostly deadweight.

Re: Writing a custom iterator in modern C++

#38

Somewhat related but C++20 ranges do not replace iterators and instead build a higher level of abstraction upon iterators. Ranges are implemented in terms of iterator pairs.

Yeah. That actually sucks; often iterators doesn't fit the iteration model of some sequences, but ranges would fit right in if they didn't require separability into two iterators.

Since C++20, the end iterator doesn't have to be the same type. Instead it can be something like a tag with no own runtime data, and comparing with it can be implemented as asking the other iterator if it is 'out of data'.

Re: Writing a custom iterator in modern C++

#39

The classic iterator categories (input, output, ...) discussed in this article have been deprecated in C++20, and replaced with new concepts. I would expect a post from 2020 claiming to be about iterators "in modern C++" to be specifically about that change, but actually it's just about the legacy categories, so this is a pretty worthless article.

On the contrary, this article can be used by the audience in almost any modern C++ compiler out there.

An article using C++20 will just lead to frustration, as there are still rough edges using concepts, with a standard that was just published last week.

Re: Writing a custom iterator in modern C++

#40
post #36

Earlier quoted context omitted.

Yeah. That actually sucks; often iterators doesn't fit the iteration model of some sequences, but ranges would fit right in if they didn't require separability into two iterators.

Can you give an example of such a sequence?

Any kind of generator is basically like this. Imagine one for listing the contents of a directory... maybe it looks like this:

  class DirectoryChildren
  {
    void *dir;
    void *buffer;
    size_t buffer_size;
    size_t buffer_offset;
    ...
  public:
    bool next(FileInfo *info);
  };
Bear in mind the buffer only represents a chunk of intermediate results fetched from the OS; when we reach its end, we have to request more data from the OS.

If you had to make an iterator for this, what would you do? You'd basically need to either (a) create iterators that at least duplicate the entire state of DirectoryChildren on the stack, or (b) make iterators allocate on the heap. Both of these suck, because (1) iterators are passed around and copied all over the place on the assumption that these are fundamentally cheap operations, which is an assumption that easily breaks (like here), and (2) the iterators fundamentally do not have any meaning or utility as separate entities from the range itself.

Post reply on HN