Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Ruby developers need to stop using EventMachine. It's the wrong direction."

slideshare.net

Post reply on HN
Print view View original

61–70 of 72 posts

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#61

Post by tptacek »

smegel wrote: post #36
tptacek wrote: post #22

Earlier quoted context omitted.

Sure, I'm using the Go http server. Take the example of a simple proxy to see where I'm coming from. Sure, I'm happy to have Go manage all the connections and sockets and I'm happy to spawn new goroutines for each connection and all that. But a proxy accepts an inbound connection, makes an outbound connection, and then monitors the outbound and inbound sides for data. The loop to do this with sockets could be a simpl…

> But instead, I have to spawn two more goroutines, one to "monitor" (really, read) from inbound, and one for outbound. What does Golang's select/case not do that you want?

Is there some feature of the language or the libraries that I am missing that turns a socket (err, a TCPConn) into a channel that I can read with Golang's select construction? Seriously asking. That would be awesome.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#62

Post by gchpaco »

tptacek wrote: post #56
gchpaco wrote: post #40

Earlier quoted context omitted.

From hard learned professional experience: EventMachine isn't written very well, when I say that I mean specific things like "whoever wrote the sub process handling in EventMachine wrote code that is uniquely wrong on every platform I've ever heard of." If you are not familiar with this (and are curious, in some sort of macabre way), I urge you do grep for SIGCHLD or wait in the source and see what you find. What Eve…

Nobody uses EventMachine to manage daemon processes. Subprocesses in EventMachine aren't "equated to popen"; they exist for the sole purpose of doing evented I/O popen-style. I'd be careful about calling a developer "incompetent" because they write something that doesn't admit to arbitrary use cases.

I'm being elliptic about why here because I don't think I can talk about the internal architecture of that system in public, but warning people off one particularly stupid third party bug that we fixed in our internal fork is not, I believe, a problem. Anyway, we certainly did use it to manage daemon processes, although not deliberately; we had a daemon that communicated with external software about system events, and running shell scripts was part of that. We didn't necessarily anticipate folks running 'service httpd start' in those shell scripts, but it was not an inherently unreasonable thing to do.

And this isn't "arbitrary use cases"; this is an explicitly supported function that is completely contrary to good practice and sane behavior and, to boot, has the ability to arbitrarily kill programs for impenetrable reasons and block for significant periods of time (the central sin of event driver programming). You can't tell me that if you saw something like this in a random crypto library you wouldn't immediately tell everybody to stop using it; why should EM's developers get a pass for their, yes, incompetently written popen? I would actually be considerably happier if it wasn't in the library at all; at least then it wouldn't be wrong.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#63

Post by tptacek »

gchpaco wrote: post #62
tptacek wrote: post #56

Earlier quoted context omitted.

Nobody uses EventMachine to manage daemon processes. Subprocesses in EventMachine aren't "equated to popen"; they exist for the sole purpose of doing evented I/O popen-style. I'd be careful about calling a developer "incompetent" because they write something that doesn't admit to arbitrary use cases.

I'm being elliptic about why here because I don't think I can talk about the internal architecture of that system in public, but warning people off one particularly stupid third party bug that we fixed in our internal fork is not, I believe, a problem. Anyway, we certainly did use it to manage daemon processes, although not deliberately; we had a daemon that communicated with external software about system events, an…

Do you have other examples of how badly constructed EventMachine is, or is it just that you can't use their process I/O stuff as a daemon manager?

I was using Adam Langley's net/ssl code in Golang to build an HTTPS proxy, and only after several hours of hair-pulling did I discover that Langley hadn't implemented the compat SSL2 handshake that Firefox uses with proxies. net/ssl in Go was, for no good reason other than an omission, unsuited for use as an HTTPS proxy. Should I say net/ssl was incompetently written? That seems like a bad idea to me.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#64

Post by tptacek »

aaronblohowiak wrote: post #25
tptacek wrote: post #9

Earlier quoted context omitted.

There are times when you might reach for select() not to scale the whole program, but because for instance you're trying to hot potato data from a Reader to a Writer. Obviously, the whole of Golang's concurrency model is lightweight threads scheduled on I/O events. But that's not exposed to the programmer; in fact, it's hermetically sealed away from the programmer from what I can tell. So now you know what I meant by…

Would you just select { case I know this isn't the same as posix select, but it does let you have one goroutine coordinate the hot potato..

That would require a goroutine for each socket to hot potato from the socket to a channel, right?

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#65

Post by smegel »

tptacek wrote: post #61
smegel wrote: post #36

Earlier quoted context omitted.

> But instead, I have to spawn two more goroutines, one to "monitor" (really, read) from inbound, and one for outbound. What does Golang's select/case not do that you want?

Is there some feature of the language or the libraries that I am missing that turns a socket (err, a TCPConn) into a channel that I can read with Golang's select construction? Seriously asking. That would be awesome.

No, I think you would need to wrap the TCPConn in a goroutine that does the Read, handles any errors, then passes the bytes back to the consumer via a channel. At least it would be a very generic and small goroutine to do this.

I agree that would be a nice feature though.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#66

Post by tptacek »

smegel wrote: post #65
tptacek wrote: post #61

Earlier quoted context omitted.

Is there some feature of the language or the libraries that I am missing that turns a socket (err, a TCPConn) into a channel that I can read with Golang's select construction? Seriously asking. That would be awesome.

No, I think you would need to wrap the TCPConn in a goroutine that does the Read, handles any errors, then passes the bytes back to the consumer via a channel. At least it would be a very generic and small goroutine to do this. I agree that would be a nice feature though.

You know what's happening in this thread? I made it sound like I was criticizing Go for needing to spawn goroutines to do this. I'm not! It makes sense, in the context of Go, to do it this way, even though as a C programmer by training that's not my first thought on how to do it.

I'm not criticizing Go for being anti-event; I'm just observing that it is. Idiomatic Go --- like, the code in the standard library --- has a strong bias towards straight-line code.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#67

Post by tptacek »

agentS wrote: post #35
tptacek wrote: post #4

We write a lot of EventMachine code here; I have some questions. He writes: EventMachine is • A frankenstein guts the ruby internals • Not in active development • Makes non-blocking IO block • Requires special code from Ruby libraries • Hard to use in an OOP way • Is really difficult to work with • Poorly documented I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't watch the talk). It's tr…

> I try to force myself to write socket code like I did when I was 13, reading a line, parsing it, and writing its response, but that code is brittle and harder to follow than a sane set of handler functions. Do like the http library does and separate out socket handling to be in terms of net.Listeners/net.Conns, and create a handler abstraction for yourself. Your socket code will be testable, because its trivial to…

I ended up writing my proxy this way. See the back-to-back "go copy" lines? That's what blew my mind. I get why Go works this way, but wow would I ever not write code that way in a threaded C program.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#68

Post by smegel »

tptacek wrote: post #66
smegel wrote: post #65

Earlier quoted context omitted.

No, I think you would need to wrap the TCPConn in a goroutine that does the Read, handles any errors, then passes the bytes back to the consumer via a channel. At least it would be a very generic and small goroutine to do this. I agree that would be a nice feature though.

You know what's happening in this thread? I made it sound like I was criticizing Go for needing to spawn goroutines to do this. I'm not! It makes sense, in the context of Go, to do it this way, even though as a C programmer by training that's not my first thought on how to do it. I'm not criticizing Go for being anti-event; I'm just observing that it is. Idiomatic Go --- like, the code in the standard library --- has…

> I'm not criticizing Go for being anti-event; I'm just observing that it is. Idiomatic Go --- like, the code in the standard library --- has a strong bias towards straight-line code.

I think I understand where you are coming from now - I think you are saying Go is "anti-event-based-callback-driven" rather than "anti-event-loop-implementation" - which is absolutely true. Go's concurrency model is build on CSP (Hoare's Communicating Sequential Processes)[1] which seems to advocate procedural threads rather than callbacks.

[1] http://golang.org/doc/go_faq.html#goroutines

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#69

Post by aaronblohowiak »

tptacek wrote: post #64
aaronblohowiak wrote: post #25

Earlier quoted context omitted.

Would you just select { case I know this isn't the same as posix select, but it does let you have one goroutine coordinate the hot potato..

That would require a goroutine for each socket to hot potato from the socket to a channel, right?

Yes. This allows you to have the "when any of these socket's state changes" semantics in your controller/dispatcher function. I would like it if they had such an interface in stdlib...

If you really do just need to take data from one Reader and send it to a Writer, you can make a new Pipe.

Post reply on HN