Why should I have written ZeroMQ in C, not C++ (2012) • Hacker News Board

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#161

Post by steveklabnik » Wed, Apr 22, 2020, 1:53 PM UTC

fluffything wrote: post #160

steveklabnik wrote: post #159

Earlier quoted context omitted.

> for absolutely no added value (doing that isn't a useful thing to do). You yourself said that web servers are a place where this is a good idea. > That doing that was a good idea was known (CERT C++ requires non-throwing destructors), and C++ actually made a backward incompatible change in C++11 to fix that (making destructors `noexcept(true)` by default, e.g., see http://www.open-std.org/jtc1/sc22/wg21/docs/papers…

> You yourself said that web servers are a place where this is a good idea. I said that "panicking from Drop::drop is a bad idea, but that panicking is a good idea in general (e.g. in web servers)". We are in agreement that panicking is a good idea in general, but you seem to be turning that around, arguing that "Panicking is a good idea in general, therefore panicking from Drop::drop is a good idea". One does not fo…

> If you believe that unwinding from Drop::drop is a good idea, enumerate the value this feature adds, its costs, and make a case about why this trade-off is worth it.

I don't believe it's a good idea, which I have repeated in this thread numerous times. You seem to keep implying that it's an often used and good idiom, and I keep saying that it's not, which is why it's not used. Most of your points are about unwinding generally, not unwinding from Drop. I think it's really confusing the issue here.

I don't really think this is productive. It seems we're in agreement about one thing, though I am very confused as to how. Let's just leave it at that.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#162

Post by fluffything » Wed, Apr 22, 2020, 2:34 PM UTC

steveklabnik wrote: post #161

fluffything wrote: post #160

Earlier quoted context omitted.

> You yourself said that web servers are a place where this is a good idea. I said that "panicking from Drop::drop is a bad idea, but that panicking is a good idea in general (e.g. in web servers)". We are in agreement that panicking is a good idea in general, but you seem to be turning that around, arguing that "Panicking is a good idea in general, therefore panicking from Drop::drop is a good idea". One does not fo…

> If you believe that unwinding from Drop::drop is a good idea, enumerate the value this feature adds, its costs, and make a case about why this trade-off is worth it. I don't believe it's a good idea, which I have repeated in this thread numerous times. You seem to keep implying that it's an often used and good idiom, and I keep saying that it's not, which is why it's not used. Most of your points are about unwindin…

> You seem to keep implying that it's an often used and good idiom,

I said that this is code that safe Rust allows you to write [0] - it's actually safe code that beginners could write by accident [1] - and therefore all unsafe Rust programmers need to actually write Rust code that defends against this happening: otherwise their safe Rust APIs over unsafe Rust code are unsound, because if someone were to write safe Rust programs like [1], their unsafe Rust code would exhibit undefined behavior (the standard library and the language itself being the prime examples of this [2]).

The claim that panicking from Drop::drop will often cause a double-drop or terminate the program assumes that all unsafe code is written by people that properly defend all their unsafe code against it happening. We agree that no reasonable programmer would purportedly write such Drop impls, so I doubt that. It suffices for execution to reach a path that isn't properly guarded for the program to exhibit UB, and at that point, there aren't any guarantees about double-drops panicking or aborts doing anything meaningful.

That is, we have a feature we agree on does not do anything useful, yet adds a cost to all unsafe code.

[0] As opposed to C++ or D, which do not allow you to write Drop::drop implementations that unwind/fail, and therefore their programmers do not need to write code to protect themselves from something that nobody does, just in case somebody actually ends up doing it by accident.

[1] As easy as:

    struct S; 
    impl Drop for S { 
        fn drop(&mut self) { unimplemented!() } 
    }

[2] These safe code example are actually used by the standard library test suite to test, e.g., the collections. So it isn't just the cost of having to read/write unsafe code that protects against something that does not make sense doing, but also the cost of then adding and maintaining the tests for those code paths.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#163

Post by fluffything » Wed, Apr 22, 2020, 2:59 PM UTC

jynelson wrote: post #155

nine_k wrote: post #140

Earlier quoted context omitted.

Look how Erlang handles this. Crashing on error is the encouraged policy. A managing process will notice a crashed process and restart it. Basically crashing is the safe way to release all resources in a problematic situation, at the cost of terminating the process. It's easy when you don't have shared resources at all (Erlang's case), and harder with threads: one threads crashes and frees its resources, another trie…

I think we're saying the same thing from two different perspectives :) The 'managing process' is the daemon thread. The 'crashed process' is a worker thread. There's no need to worry about corrupted memory since all state is shared through the database.

The main practical difference is that if you use a process for isolation, the OS will garbage collect all process resources on termination (memory, file descriptors, threads, mutexes, etc.).

If you are using a thread, you better not be leaking anything. Otherwise you are "ulimits" resources away from putting your "main" process into a crashing loop, e.g., if you leak 1 file-descriptor per crash, then you can crash your thread Performance-wise, you are probably worse with threads as well. On linux, you can initialize a web-server on your main process, and spawn new processes by forking it. Forking isn't only pretty much instantaneous, your child process is initialized with the same state as the parent, so you instantaneously get a fully initialized web server (e.g. with multiple threads already started in your task pool, etc.).

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#164

Post by CodesInChaos » Wed, Apr 22, 2020, 4:18 PM UTC

steveklabnik wrote: post #156

CodesInChaos wrote: post #154

Earlier quoted context omitted.

> It will be leaked if panic=unwind and if you use catch_panic. An uncaught panic terminates the thread. If that's the main thread the process terminates, but on other threads it continues with leaked memory, even without `catch_unwind`. > catch_panic is not used very much. Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic. Otherwise any bug causing a panic (common in my e…

> If that's the main thread the process terminates, but on other threads it continues with leaked memory Yep, my bad, I simply made a mistake here. > Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic. The two main use-cases for panics seem to be: 1. web servers 2. FFI, since panic across the boundary is UB On 1, well, this is actually a contentious point. In general, the w…

A reproducible panic can easily take down your whole fleet of servers, so you'd end up with a one process per request model without planning for this.

Plus if a webserver handles more than one request in parallel (affects both multi-threaded and node.js style async designs), you want to finish handling all the other requests before shutting down gracefully.

In my experience with C# even exceptions caused by bugs/failed assertions (which would map to panics in rust) almost never cause issues which require a server restart. It's a bit worse in Rust, since a GC collects memory more reliably in case of errors than Rust (but even in Rust leaks should rarely happen on panic)

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#165

Post by jynelson » Wed, Apr 22, 2020, 5:05 PM UTC

fluffything wrote: post #163

jynelson wrote: post #155

Earlier quoted context omitted.

I think we're saying the same thing from two different perspectives :) The 'managing process' is the daemon thread. The 'crashed process' is a worker thread. There's no need to worry about corrupted memory since all state is shared through the database.

The main practical difference is that if you use a process for isolation, the OS will garbage collect all process resources on termination (memory, file descriptors, threads, mutexes, etc.). If you are using a thread, you better not be leaking anything. Otherwise you are "ulimits" resources away from putting your "main" process into a crashing loop, e.g., if you leak 1 file-descriptor per crash, then you can crash yo…

Hmm, this is an interesting point about the OS cleaning up resources. I'll see if it's feasible to switch from threads to processes. Thanks!

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#166

Post by zwieback » Wed, Apr 22, 2020, 7:09 PM UTC

mehh wrote: post #111

gumby wrote: post #81

Earlier quoted context omitted.

I decided to treat it as if it was a language I had never seen before. That helped a lot.

Likewise, but also I now realise, after a long break I really like programming with "system languages". I get a certain amount of satisfaction that I don't get with higher-level languages, where I spend so much time plugging together half-built crappy modules. Partly this is because I have realised, I like knowing the nuts and bolts of what is going on in a computer and solving problems with consideration of how it w…

Totally agree. I enjoy the productivity in Python but when you look what's behind numpy, scipy, etc. there's a lot of C, C++ and even Fortran. It's good to know both.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#167

Post by hedora » Thu, Apr 23, 2020, 3:24 PM UTC

Given the article’s age, the issues brought up are reasonable.

Now that it’s 8 years later, I’d summarize it as:

- exceptions were a bad idea (in any language); don’t use them.

- implementation inheritance and constructors / destructors that contain non-trivial logic were a bad idea; don’t use them.

This basically means you shouldn’t be writing idiomatic object oriented code.

Fortunately, C++ supports other programming paradigms, and in current C++, you can avoid both these historical warts; they shouldn’t affect day-to-day systems programming or error handling.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#168

Post by hedora » Thu, Apr 23, 2020, 3:35 PM UTC

sxyuan wrote: post #6

It seems like the author's main complaints are avoided if one follows the Google C++ style guide, which says: - Don't use C++ exceptions - Don't do work in constructors (prefer "Init" or factory functions instead) https://google.github.io/styleguide/cppguide.html There may be other reasons to prefer C over C++, but if you don't like exceptions, you don't have to use them.

Regardless of what the style guide says, multiple studies of real-world exception handling correctness have been done, and the results are clear: Very few (if any) real-world codebases correctly handle exceptions.

As the article points out, correct exception handlers would grow exponentially with program size. People can’t reason about that, so they don’t handle things like multiple error paths, or they conflate one error code with another, and so on.

The results hold across programming languages.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#169

Post by Supermancho » Thu, Apr 23, 2020, 10:22 PM UTC

mycelium wrote: post #142

Supermancho wrote: post #128

Earlier quoted context omitted.

Using ZMQ (from C) at high volume, we found it dropped data, so we abandoned it.

What did you replace it with?

Direct writes locally to ebs and then stream to S3 and async collation after.

Re: Why should I have written ZeroMQ in C, not C++ (2012)

#170

Post by mehh » Fri, Apr 24, 2020, 5:42 PM UTC

zwieback wrote: post #166

mehh wrote: post #111

Earlier quoted context omitted.

Likewise, but also I now realise, after a long break I really like programming with "system languages". I get a certain amount of satisfaction that I don't get with higher-level languages, where I spend so much time plugging together half-built crappy modules. Partly this is because I have realised, I like knowing the nuts and bolts of what is going on in a computer and solving problems with consideration of how it w…

Totally agree. I enjoy the productivity in Python but when you look what's behind numpy, scipy, etc. there's a lot of C, C++ and even Fortran. It's good to know both.

Had a brief look at boost-python (not actually done anything with it), and that looks quite straight forward for exposing C++ to Python.