Live data from Hacker News

"Ruby developers need to stop using EventMachine. It's the wrong direction."

slideshare.net

51–60 of 72 posts

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#51
Well, when I hear about Actors I reach for my gun.

I knew a guy who chose Scala for a project so he could use Actors for concurrency.

The system never gave the same answers twice and wouldn't peg all the cores on a 4-way machine.

I spent two days trying to fix it, then I got wise and switched back to Java and got it working in 20 minutes with ExecutorService with (i) no race conditions, and (ii) nearly perfect scaling up to eight cores.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#52

There's a fork out there called EventMachine-LE, which is currently in active development for latest features (i.e. IPv6). https://github.com/ibc/EventMachine-LE Anyway, just wanted to throw it out there that I've deployed several production apps (gaming/messaging servers) using EventMachine, and combined with em-synchrony, I'm pretty comfortable with its performance and limitations. There seems to be good community…

Sidekiq is probably the best example of Celluloid in action.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#54
post #30

Not to judge this presentation up or down; I'm not a Rubyist... but a reflection for myself - you know you're getting old when you see ideas that have come and gone and come and gone come again.

Like rings on a tree, you can judge how old you are by how many times a particular idea has resurfaced after some time in obscurity.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#55
post #24

If you really like evented programming, use node. Everything there is evented. I tried EventMachine and it just felt worse than node, documentation is worse, libraries are worse, it just wasn't a whole lot of fun. I haven't tried celluloid yet, but I've learned one lesson building things with PHP, Ruby, Python, Node, Java, Scala, etc. That lesson is to use tools with a community of people around it that are using tho…

Python programmers have been doing evented code for almost a decade using Twisted. EventMachine happened later, but then, so did the mainstreaming of Ruby. You realize, don't you, that Tcl has all three of Javascript, Ruby, and Python beat when it comes to evented I/O? Node does not own evented I/O.

I agree that Node doe not own evented I/O, my point was more that if you care about concurrency, parallelism, evented io, you should work in communities that care about those things and build tools around them. Ruby as a community cares more about Rails than evented I/O. Node is purely evented I/O, so that's basically ALL they care about. I guess Python is probably more in between.

I didn't mean to say node owns evented I/O, just that their whole community embraces it.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#56
post #40
post #4

We write a lot of EventMachine code here; I have some questions. He writes: EventMachine is • A frankenstein guts the ruby internals • Not in active development • Makes non-blocking IO block • Requires special code from Ruby libraries • Hard to use in an OOP way • Is really difficult to work with • Poorly documented I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't watch the talk). It's tr…

From hard learned professional experience: EventMachine isn't written very well, when I say that I mean specific things like "whoever wrote the sub process handling in EventMachine wrote code that is uniquely wrong on every platform I've ever heard of." If you are not familiar with this (and are curious, in some sort of macabre way), I urge you do grep for SIGCHLD or wait in the source and see what you find. What Eve…

Nobody uses EventMachine to manage daemon processes. Subprocesses in EventMachine aren't "equated to popen"; they exist for the sole purpose of doing evented I/O popen-style. I'd be careful about calling a developer "incompetent" because they write something that doesn't admit to arbitrary use cases.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#57
post #26
post #4

We write a lot of EventMachine code here; I have some questions. He writes: EventMachine is • A frankenstein guts the ruby internals • Not in active development • Makes non-blocking IO block • Requires special code from Ruby libraries • Hard to use in an OOP way • Is really difficult to work with • Poorly documented I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't watch the talk). It's tr…

When I saw the title, I thought it was going to be coverage of how under-the-hood EventMachine's model only allows it to scale up to a certain throughput per-process (which is reasonably high for most uses of it) or how there is some fundamental complexity in how it's written that pretty much guarantees there will be livelock conditions (and deadlock conditions). So I share your bewilderment about why they chose the…

You're threading, I presume? We do high speed high connection rate work in EventMachine (for instance, to work with order entry for trading exchanges) and have never seen anything like this --- but we religiously avoid Ruby threads.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#58
post #47
post #19

Earlier quoted context omitted.

> I'm not sure I understand how EventMachine "guts the Ruby internals" EventMachine does not use the Ruby IO primitives (e.g. TCPSocket, UDPSocket, etc) as the basis of its IO abstraction, and instead has reimplemented its own set of primitives for doing IO. Because of this it can't take advantage of work being done in Ruby core to advance Ruby's socket layer. For this reason IPv6 support langered, among other proble…

At least 2 years ago, twisted did not use libevent, it was mostly pure python except for a very tiny piece of code (and optional stuff of course). I am sure you could write a libevent-based reactor, but twisted was started a long time ago (10 years ? The twisted book in o'Reilly was published in 2005), before libevent existed I think.

No, libevent existed before Twisted; its first release was in 2000, and we used it extensively at Arbor Networks just a couple years later.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#60
post #57
post #26

Earlier quoted context omitted.

When I saw the title, I thought it was going to be coverage of how under-the-hood EventMachine's model only allows it to scale up to a certain throughput per-process (which is reasonably high for most uses of it) or how there is some fundamental complexity in how it's written that pretty much guarantees there will be livelock conditions (and deadlock conditions). So I share your bewilderment about why they chose the…

You're threading, I presume? We do high speed high connection rate work in EventMachine (for instance, to work with order entry for trading exchanges) and have never seen anything like this --- but we religiously avoid Ruby threads.

Yeah we are, as it was the least-convoluted method for handling the multiplexing (but wanting to maintain the same handler for all of them). Burned entirely too much time assuming that documented ways to handle things were actually rock solid, not rarely used. The first bug where 3 connections could deadlock everything was unrelated to threading. It was related to how it created identifiers for each connection and would create identical identifiers for two different connections so it would think there was data pending for a socket, but the data had already been read, so it tried to do a blocking read when there wasn't data to read. I never fully tracked down the second bug, but it was more likely related to using threading.

One thing we did have was a large majority of the connections were coming from the same IP (but different source port) and a peak connection rate of ~200 connections per second per server and would last a few seconds (so at any given moment, roughly 1000 connections being established or with data in flight). I'd be really interested in hearing what your traffic patterns are roughly like (and hand-wavy what you have event machine doing, like proxying requests, building/returning its own responses, etc.) if you're willing to share at all.

Post reply on HN