Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Ruby developers need to stop using EventMachine. It's the wrong direction."

slideshare.net

Post reply on HN
Print view View original

11–20 of 72 posts

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#11

Post by ericb »

tptacek wrote: post #7
danielszm wrote: post #5

Earlier quoted context omitted.

I wonder why nobody ever talks about drb, a distributed system that ships with ruby. It's active in Japan, and there has been an effort to make it better known in the rest of the world with a book dedicated to the topic in english (published by Pragmatic Programmers). Still, it hasn't stirred much interest. Even if it's less advanced than DCell, it's still a recommended way to start learning about distributed computi…

It's brittle, opaque, and relies on Marshal, an interchange standard nothing else uses. It's so easy to build alternatives to Drb that use better interchanges that nobody uses Drb. And once you do that, you find it's easy to back your system with Redis or a database, which you do because it makes sense, and then you start getting pulled towards message architectures --- which are themselves usually superior to direct…

In my previous load testing of Drb, at 86 connections (remote objets) with even moderate use, things consistently went bad. This was independent of number of machines.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#12

Post by danielszm »

tptacek wrote: post #7
danielszm wrote: post #5

Earlier quoted context omitted.

I wonder why nobody ever talks about drb, a distributed system that ships with ruby. It's active in Japan, and there has been an effort to make it better known in the rest of the world with a book dedicated to the topic in english (published by Pragmatic Programmers). Still, it hasn't stirred much interest. Even if it's less advanced than DCell, it's still a recommended way to start learning about distributed computi…

It's brittle, opaque, and relies on Marshal, an interchange standard nothing else uses. It's so easy to build alternatives to Drb that use better interchanges that nobody uses Drb. And once you do that, you find it's easy to back your system with Redis or a database, which you do because it makes sense, and then you start getting pulled towards message architectures --- which are themselves usually superior to direct…

Happy to see informed opinions, thank you. Sounds pretty damning, too. Thomas, are you saying that messaging is always superior to RMI, and that there is no valid use case for RMI? I am thinking out loud here, but isn't a distributed system that leverages RMI quicker to implement? Might be suitable when prototyping. Or teaching a classroom. Does that make sense?

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#13

Post by zemo »

tptacek wrote: post #9
stock_toaster wrote: post #6

Earlier quoted context omitted.

> Golang is militantly anti-event; it doesn't even offer a select primitive! Just alternating read/write on two different sockets seems to demand threads! Go only blocks the goroutine. It uses native event primitives (epoll/kqueue/etc) under the hood so that IO is non-blocking to the rest of the process (e.g. other goroutines). You said you have used Go for 2 months, so I assume you are aware of that. So..can you be…

There are times when you might reach for select() not to scale the whole program, but because for instance you're trying to hot potato data from a Reader to a Writer. Obviously, the whole of Golang's concurrency model is lightweight threads scheduled on I/O events. But that's not exposed to the programmer; in fact, it's hermetically sealed away from the programmer from what I can tell. So now you know what I meant by…

every time you perform i/o, the current goroutine yields to the scheduler.

If you want to manually yield to the scheduler, you call runtime.Gosched() and the calling goroutine yields. If you want to lock a goroutine into a specific thread, you call runtime.LockOSThread(). Everything is single-threaded to start until you call runtime.GOMAXPROCS(), but that's intended to go away in the very near future. So... I'm not sure what you mean by sealed away; the Go runtime does the sensible things that it can do automatically, but you still have the ability to change the scheduling behavior if you really want to.

what do you mean by "doesn't offer a select primitive"? I'm not familiar with your definition of select, because Go has a select keyword for concurrency control, and I'm under the impression you're talking about select as its defined in EventMachine, could you elaborate?

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#14

Post by burke »

ericb wrote: post #2

I've looked at Celluloid. It offers a much nicer paradigm than eventmachine. It also provides a nice way to distribute actors over multiple machines (DCell). I strongly agree with this slidedeck. Callback muck is maddening.

As an added bonus, every time I've tried a problem in both, Celluloid has been faster.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#16

Post by bascule »

danielszm wrote: post #5
ericb wrote: post #2

I've looked at Celluloid. It offers a much nicer paradigm than eventmachine. It also provides a nice way to distribute actors over multiple machines (DCell). I strongly agree with this slidedeck. Callback muck is maddening.

I wonder why nobody ever talks about drb, a distributed system that ships with ruby. It's active in Japan, and there has been an effort to make it better known in the rest of the world with a book dedicated to the topic in english (published by Pragmatic Programmers). Still, it hasn't stirred much interest. Even if it's less advanced than DCell, it's still a recommended way to start learning about distributed computi…

Hello, author of Celluloid here.

I think DRb pretty cool and has long been underutilized. That said, there are a few fundamental design problems with the way it works that I think DCell solves:

1. Distributed systems really need to be built on top of asynchronous protocols, and DRb is a direct mapping of a synchronous method dispatch protocol onto a distributed systems protocol. Similar attempts at this include: CORBA and SOAP. If anyone disagrees with this I can go into more detail but I think you will find ample distributed systems literature condemning synchronous protocols. DCell is fully asynchronous (but also provides synchronous calls over an underlying asynchronous protocol)

2. DRb is multithreaded but does not provide the user with any assistance in building multithreaded programs. This becomes particularly confusing when you have to deal with a proxy object (DRb::DRbObject) in a remote scenario but not in a local scenario

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#17

Post by adgar2 »

tptacek wrote: post #4

We write a lot of EventMachine code here; I have some questions. He writes: EventMachine is • A frankenstein guts the ruby internals • Not in active development • Makes non-blocking IO block • Requires special code from Ruby libraries • Hard to use in an OOP way • Is really difficult to work with • Poorly documented I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't watch the talk). It's tr…

> But there are problems --- like, backend processing, or proxies, or routers and transformers, or feed processors --- where event loops are the most natural way to express a performant solution.

Your backend processing apparently fits in one machine, since you "hook Mysql up through Redis." I'm personally astonished you get so much done without a distributed environment tolerant of the relevant failures I see when I read that sentence.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#18

Post by programminggeek »

If you really like evented programming, use node. Everything there is evented. I tried EventMachine and it just felt worse than node, documentation is worse, libraries are worse, it just wasn't a whole lot of fun.

I haven't tried celluloid yet, but I've learned one lesson building things with PHP, Ruby, Python, Node, Java, Scala, etc. That lesson is to use tools with a community of people around it that are using those tools to solve the same problems as you. The more community is around a project the more likely that someone else has stumbled upon whatever bug or problem you have hit already and found a solution if it exists. Also, documentation tends to be better and easier to find on more popular languages and frameworks.

So, if you like busting out CRUD web apps, you probably should look at PHP and Zend/Symfony/etc. framework or Ruby on Rails or Python Django or Java Play. If you like doing single page JS apps you should look at Knockout, Backbone, and Ember. If you want to do more evented/parallel networked apps you should look at node, scala, clojure, go, and erlang because those communities care a lot about threading, evented, actor pattern type programming.

Evented/multicore/multithreaded programming is just not something that say PHP, Ruby, and Python have embraced as much as a community because it's not for the most part a problem that the average PHP, Ruby, and Python dev is trying to solve most of the time.

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#19

Post by bascule »

tptacek wrote: post #4

We write a lot of EventMachine code here; I have some questions. He writes: EventMachine is • A frankenstein guts the ruby internals • Not in active development • Makes non-blocking IO block • Requires special code from Ruby libraries • Hard to use in an OOP way • Is really difficult to work with • Poorly documented I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't watch the talk). It's tr…

> I'm not sure I understand how EventMachine "guts the Ruby internals"

EventMachine does not use the Ruby IO primitives (e.g. TCPSocket, UDPSocket, etc) as the basis of its IO abstraction, and instead has reimplemented its own set of primitives for doing IO.

Because of this it can't take advantage of work being done in Ruby core to advance Ruby's socket layer. For this reason IPv6 support langered, among other problems.

This also severely complicates making multiple implementations of the EventMachine API, such as its JRuby backend (which maps onto Java NIO)

I think the real problem is EventMachine's original goal was to be a cross-language I/O backend similar to libevent or libev, but since it wasn't a particularly good one, the only language that wound up using it was Ruby. Compare to Twisted, which is built on libevent, or to Node, which is built on libev/libuv

Re: "Ruby developers need to stop using EventMachine. It's the wrong direction."

#20

Post by stock_toaster »

tptacek wrote: post #9
stock_toaster wrote: post #6

Earlier quoted context omitted.

> Golang is militantly anti-event; it doesn't even offer a select primitive! Just alternating read/write on two different sockets seems to demand threads! Go only blocks the goroutine. It uses native event primitives (epoll/kqueue/etc) under the hood so that IO is non-blocking to the rest of the process (e.g. other goroutines). You said you have used Go for 2 months, so I assume you are aware of that. So..can you be…

There are times when you might reach for select() not to scale the whole program, but because for instance you're trying to hot potato data from a Reader to a Writer. Obviously, the whole of Golang's concurrency model is lightweight threads scheduled on I/O events. But that's not exposed to the programmer; in fact, it's hermetically sealed away from the programmer from what I can tell. So now you know what I meant by…

Thanks. I think I understand a bit better now.

Go provides concurrency primitives (scheduler yielding i/o, channels, goroutines, etc) upon which you can build your own "events". I agree (and this seems to be your point) that Go doesn't provide a canned event mechanism that you can just hook into for callbacks on IO.

For example in the Go http server[1] a goroutine accepts in a loop and kicks off goroutines to process and handle new requests.

[1]: http://golang.org/src/pkg/net/http/server.go?s=30200:30241#L...

Post reply on HN