Live data from Hacker News

Dalin – A C++ non-blocking network library on Linux

github.com

31–40 of 52 posts

Re: Dalin – A C++ non-blocking network library on Linux

#31
post #5

Earlier quoted context omitted.

On the one hand, you're right -- in that I can see and respect why you say that. On the otherhand, what the heck is wrong with using the syscalls? Every time I write a new high performance networking application, sure, i lose a day or two building primitives out of send recvmsg listen accept and splice, but once i'm done, i'm done, and things work. I'm confused why people feel the need to make paper thin abstraction…

> On the one hand, you're right -- in that I can see and respect why you say that. > On the otherhand, what the heck is wrong with using the syscalls? Every time I write a new high performance networking application, sure, i lose a day or two building primitives out of send recvmsg listen accept and splice, but once i'm done, i'm done, and things work. ... > Tangentially, I positively can't WAIT for Networking suppor…

And on macOS BSD sockets are on their way out, as the new networking stack (user space based) doesn't support BSD sockets (page 15).

https://devstreaming-cdn.apple.com/videos/wwdc/2017/707h2gkb...

Re: Dalin – A C++ non-blocking network library on Linux

#32
post #18

From a short review I don't like this much. - It's containing it's own abstraction over pthread instead of std::thread/std::mutex/... - It passes shared_ptr via reference (yeah, this saves an increment and decrement on the refcounting, but the code is not tuned that much) - it prints directly to sterr (via fprintf) making it hard to redirect errors somewhere else - It seems not to be prepared to be ported to other pl…

> It passes shared_ptr via reference Talk about not getting it.

Can you explain what you mean here? Isn't what the op is doing generally ok - to pass shared_ptr's by const reference to save an additional increment/decrement on function call?

https://stackoverflow.com/questions/3310737/shared-ptr-by-re...

Re: Dalin – A C++ non-blocking network library on Linux

#34
post #7
post #6

Earlier quoted context omitted.

"asio requires boost"

Not necessary to pull whole boost

So far I've never met any clients or employers in my life who use Boost in production. How do you extract particular libraries (in a "good" way)?

EDIT: as others commented, asio is available as a stand-alone lib as well. But this is not the case for most of other parts of Boost I believe.

Re: Dalin – A C++ non-blocking network library on Linux

#36

From a short review I don't like this much. - It's containing it's own abstraction over pthread instead of std::thread/std::mutex/... - It passes shared_ptr via reference (yeah, this saves an increment and decrement on the refcounting, but the code is not tuned that much) - it prints directly to sterr (via fprintf) making it hard to redirect errors somewhere else - It seems not to be prepared to be ported to other pl…

I agree with the last sentence. Nevertheless, if it's mainly a learning project for the author, I don't think there's anything wrong with writing it.

Some technical review for the presented library:

- It doesn't seem to support any kind of backpressure, which is basically a nogo if you want to build something reliable on top. On the receiving side you will always get data pushed via callbacks without the possibility to stop it. On the sending side it will always return void and queue the data internally if it can't be send immediatly. A slow, malfunctioning or attacking remote can DOS you through that. Ways to implement backpressure are pull-style operations like in boost asio (you start a read and get a single callback when it's done) or some pause/unpause functions and buffer treshold indicators (like in node.js). I personally prefer the first model now, especially if the single operations return something like a promise.

- That brings me to the next point: The framework does not seem to have a good way to build composable operations. E.g. build a function that reads a websocket frame from the socket. Or another function that performs the websocket handshake by reading the HTTP handshake request and sending the associated response, but leaving the stream intact for any following user to to be able to use if for sending websocket frames. And ideally all operations should be trivially boundable by timeouts. With having a single receive callback for available data there's mostly a need for complex callback-driven state machines.

- Imho a new good framework should also provide a sophisticated and universal story for cancellation of composed operations. Like backpressure that's a thing which can be avoided for demo applications, but for battleproof production environments it is necessary.

- Thread safety and data races: There seem to be a few issues, e.g. TcpConnection::state isn't synchronized and used from multiple threads. loop_->runInLoop([&](){ this->shutdownInLoop(); }); will segfault or cause undefined behavior if the TcpConnection object was deleted befor the queued functor runs. There also might be some reentrancy issues: In each callback to the user (like MessageCallback) the user might fiddle around with the object and change it's internal state. If that isn't expected and guarded against then code that runs after the invocation of the callback might not work as intended. By the way: That's the situation where js-style promises shine: As the callbacks are not immediatly invoked but in the next eventloop iteration there is less rooom for errors. But of course it costs additional performance.

If these things now sound as a harsh critique let me try to bring it back into relation again. First of all: Network programming is super hard because of "concurrency everywhere". There are always some execution paths that one hadn't thought about before, and it takes a lot of time to learn all the gotchas. I do that stuff now for 8 years, probably had worse assumptions and code for quite some time, and still learn now things every day. It's for sure not a bad idea to write an own library in order to learn these things. For users the well-known libraries (asio, libuv, QT Network, etc.) might be a more solid choice.

Also in my opinion even the big and well-known libraries have some dark corners, there hasn't been a golden solution to network programming yet:

- asio is hard to use, callback that are happening on invalidated objects or operations that are still running and use invalidated buffers are are common problem for non-experts. It get's worse if you use asio with multiple threads (especially one io_service and multiple threads). Composed operations are possible but hard to write efficiently (best put all shared-state between operations in shared_ptr's and write a state machine. Maybe it's better with coroutine support.

- libuv is generally well engineered, but is still doesn't offer lots of support for higher level composed operations and cancellation.

- Netty has some dark corners to watch out for. E.g. if handlers are exchanged during runtime (e.g. for websocket upgrades) or handlers are running in multiple threads there's quite some room for errors on the user side. Otherwise it's a well engineered framework in many areas.

- node.js stream abstractions with their multiple modes can be tricky to understand and to use for higher-level abstractions. I guess if networking would be based on promises with async/await support it would be easier now - but those weren't available back then.

Imho the most promising concepts for network programming are the coroutine based ones. Go's model works well, is on the easier side to use (even though goroutines/threads introduce more possibilities for race conditions). It ticks most boxes, e.g. backpressure is naturally available through sync APIs. Cancellation could probably be improved.

Martin Sustriks experiments with libdill also look interesting. As well as Kotlins coroutine model.

The promise based models with async/await are also quite promising, e.g. in C# or node.js. However I find those models fall a little bit down as soon as they go to support both multiple threads and an event loop. Having to remember which callbacks/continuations run on which thread and what is allowed there is cumbersome. Imho either multiple threads with synchronous code or a single thread with an eventloop is ok.

Ooops, that was now a little bit more text than I intended to write. But maybe it helps someone.

Re: Dalin – A C++ non-blocking network library on Linux

#37
post #32
post #18

Earlier quoted context omitted.

> It passes shared_ptr via reference Talk about not getting it.

Can you explain what you mean here? Isn't what the op is doing generally ok - to pass shared_ptr's by const reference to save an additional increment/decrement on function call? https://stackoverflow.com/questions/3310737/shared-ptr-by-re...

It isn't idiomatic. If you've already locked the shared_ptr, you should extract it's boxed value and pass that by reference directly. Passing a shared_ptr by reference has fairly niche applications and allows the callee to do things like reset the shared_ptr, extract a weak_ptr from it, etc.

That said, if you are in need of the latter use case, it certainly should be passed by reference. It's not just a reference count! Shared pointers in C++ are threadsafe so there's a fair bit more going on under the hood that makes copying it (more) expensive.

Re: Dalin – A C++ non-blocking network library on Linux

#38
post #33

had a quick look at the src/tests, from what I can tell, it is more like a hobby project started for learning c++11/networking. a few of those tests are really more like examples

Good, because the readme is severely lacking in any samples. With a claim of "Simple API”, not providing samples in the readme is a sin.

Re: Dalin – A C++ non-blocking network library on Linux

#39

Earlier quoted context omitted.

Compile time is a pretty big reason.

so do you also not use std::unordered_map, std::shared_ptr, std::thread, std::regex, or std::mutex ? because they all come from boost.

Once they're safely out of Boost and implemented by the stdlib they're an order of magnitude more convenient to use. You can also be more confident in their future stability.

Re: Dalin – A C++ non-blocking network library on Linux

#40
post #26
post #24

Earlier quoted context omitted.

Given the universality of C++, everyone effectively ends up writing in their own dialect and there is a sizeable faction that uses it as "C with Classes" and "C with templates". So there is a lot of C++ projects that explicitly steer clear of modern C++ features, especially derived from the mother of all unholy abominations and the manifestation of everything that is wrong with the modern C++, the boost library. Give…

Usually those are the projects where CVEs are a common feature..

... as are the projects that overdo on "modern C++" to the extent of introducing even more severe problems masked by the layers of needless abstraction.

CVEs don't come from C or C++. They come from the lack of coding discipline and with all other things being equal C programmers are generally better skilled and more diligent than their C++ counterparts.

Post reply on HN