Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we used Pony to write Wallaroo

blog.wallaroolabs.com

Post reply on HN
Print view View original

51–60 of 87 posts

Re: Why we used Pony to write Wallaroo

#51

Post by 59nadir »

spooneybarger wrote: post #48
59nadir wrote: post #34

Earlier quoted context omitted.

> Biggest weakness would be maturity. Has there been any progress towards a preemptive scheduler? Otherwise, I'd think that's the most obvious weakness.

There's pros and cons to preemptive and cooperative scheduling policies. I wouldn't be comfortable labelling either as weakness. It really depends on context. I've heard rumor of someone working on a preemptive scheduler. The current cooperative one is about to have runtime backpressure added to it which is going to be a really nice scheduling win. https://github.com/ponylang/ponyc/pull/2264

For the kinds of systems that are built with Erlang, which is a niche that Pony seems to want to occupy, I would argue it's inherently better to have preemptive scheduling. A cooperative scheduler will always be an open invitation for bugs relating to CPU hogging. Not having to worry about these things is priceless.

Re: Why we used Pony to write Wallaroo

#52

Post by pdimitar »

scottlf wrote: post #47
model_m_warrior wrote: post #20

Earlier quoted context omitted.

Are Ponys classes akin to Erlang modules or does it take a more enterprise-y perspective on OOP?

Howdy. Erlang's modules are involved in code namespace for the compiler & runtime (i.e., the 'M' of the 'MFA' triple of Module + Function name + Arity that names a function) as well being the unit/scope/bound for hot code loading (i.e., you must load or unload an entire module at a time). I've worked in Erlang-land far longer than I've lived in any OOP-land, so I'm not sure what you mean by enterprise'y OOP. Coincide…

I'll take the liberty to speculate and assume your parent comment was asking if Pony classes have the same features as Java classes -- namely class-level attributes, instance-level attributes, different access levels to attributes and methods. Those things.

So, does it?

Re: Why we used Pony to write Wallaroo

#53

Post by jdp23 »

spooneybarger wrote: post #2

Author here. Small bit of background. I'm VP of Engineering at Wallaroo Labs and a member of the Pony core team (many folks at Wallaroo Labs are now actively involved in the Pony community). Happy to answer any questions here or if you prefer, via email: sean@wallaroolabs.com

Just wanted to say, thanks for the excellent article and the answers to questions here!

Re: Why we used Pony to write Wallaroo

#54

Post by MrBuddyCasino »

spooneybarger wrote: post #2

Author here. Small bit of background. I'm VP of Engineering at Wallaroo Labs and a member of the Pony core team (many folks at Wallaroo Labs are now actively involved in the Pony community). Happy to answer any questions here or if you prefer, via email: sean@wallaroolabs.com

I‘ve worked with Scala/Akka, and while I like the actors model, some things turn me off about the stack, mostly the language.

After reading the Pony guiding principles, I was delighted. This looks like an implementation where the ecosystem complexities don’t get in my way of „getting things done“!

Re: Why we used Pony to write Wallaroo

#55

Post by spooneybarger »

jdp23 wrote: post #53
spooneybarger wrote: post #2

Author here. Small bit of background. I'm VP of Engineering at Wallaroo Labs and a member of the Pony core team (many folks at Wallaroo Labs are now actively involved in the Pony community). Happy to answer any questions here or if you prefer, via email: sean@wallaroolabs.com

Just wanted to say, thanks for the excellent article and the answers to questions here!

Thank you!

Re: Why we used Pony to write Wallaroo

#56

Post by spooneybarger »

jack9 wrote: post #42
spooneybarger wrote: post #2

Author here. Small bit of background. I'm VP of Engineering at Wallaroo Labs and a member of the Pony core team (many folks at Wallaroo Labs are now actively involved in the Pony community). Happy to answer any questions here or if you prefer, via email: sean@wallaroolabs.com

How do you ingest data with Wallaroo? I see a mention of "...say TCP" and in the github word count there is mention of a (data)source framed-message-protocol but nothing that describes it in the ( https://github.com/WallarooLabs/wallaroo/blob/0.2.0/book/cor... ) link. Dropping Python into the mix for a high throughput processing pipeline seems counterproductive. Why isn't there a tutorial in a more strict language li…

Ingestion:

Kafka and framed TCP are the two ingestion sources that Wallaroo currently ships with.

Python:

We added a Python API because there was a lot of interest in it. There's more to Wallaroo than just performance and folks were interested in having it available via Python. See https://vimeo.com/234753585 for some more information on the "scale independent" nature of the Wallaroo APIs.

Go:

We're working on that right now actually. Planning to release later this year.

C++:

We had a C++ API (still do), we aren't currently supporting it. There was limited interest at the time. If folks show interest we would start supporting it again.

Re: Why we used Pony to write Wallaroo

#57

Post by spooneybarger »

pdimitar wrote: post #52
scottlf wrote: post #47

Earlier quoted context omitted.

Howdy. Erlang's modules are involved in code namespace for the compiler & runtime (i.e., the 'M' of the 'MFA' triple of Module + Function name + Arity that names a function) as well being the unit/scope/bound for hot code loading (i.e., you must load or unload an entire module at a time). I've worked in Erlang-land far longer than I've lived in any OOP-land, so I'm not sure what you mean by enterprise'y OOP. Coincide…

I'll take the liberty to speculate and assume your parent comment was asking if Pony classes have the same features as Java classes -- namely class-level attributes, instance-level attributes, different access levels to attributes and methods. Those things. So, does it?

Pony classes have methods. Pony classes have fields.

Fields and methods can be public or private. Private is akin to Java's package private.

There are no instance variables.

Re: Why we used Pony to write Wallaroo

#58

Post by switchbak »

polskibus wrote: post #39
aturley wrote: post #29

Earlier quoted context omitted.

Pony does not do preemptive scheduling. Pony actors have behaviors, which can be thought of as methods that handle messages that are sent to the actor; these are the unit of scheduling, and a behavior runs from beginning to end without preemption. Multiple behaviors can run at the same time, but only one behavior per actor can be running at any time. There is currently no equivalent to Distributed Erlang, but that's…

Preemptive scheduling is one of the advantages of Erlang over other actor implementations like Akka. Please consider implementing it!

Admittedly I'm not familiar with Erlang's preemptive model, but I'll ask anyway :)

When using async based APIs with something like Akka, passing around futures and such, and leveraging multiple actors in thread pools, doesn't that get you to approximately the same place as Erlang?

Sure, the JVM isn't doing the preempting, but the OS is. Is the main downside per-thread resource consumption and context scheduling overhead?

Re: Why we used Pony to write Wallaroo

#59

Post by anko »

switchbak wrote: post #58
polskibus wrote: post #39

Earlier quoted context omitted.

Preemptive scheduling is one of the advantages of Erlang over other actor implementations like Akka. Please consider implementing it!

Admittedly I'm not familiar with Erlang's preemptive model, but I'll ask anyway :) When using async based APIs with something like Akka, passing around futures and such, and leveraging multiple actors in thread pools, doesn't that get you to approximately the same place as Erlang? Sure, the JVM isn't doing the preempting, but the OS is. Is the main downside per-thread resource consumption and context scheduling overh…

I think there is a lot to be said for the simplicity of the erlang model. It's not just about stopping a thread from hogging the cpu. The OS doesn't have a good idea when to wake threads up, so they are effectively polling needlessly.

I do scala/akka in my dayjob and the amount of time we spend tweaking threadpools (execution contexts) and retesting just to get okay performance is insane. You wear a pretty big cost when you layer on abstractions like that.

Re: Why we used Pony to write Wallaroo

#60

Post by Cyph0n »

MrBuddyCasino wrote: post #54
spooneybarger wrote: post #2

Author here. Small bit of background. I'm VP of Engineering at Wallaroo Labs and a member of the Pony core team (many folks at Wallaroo Labs are now actively involved in the Pony community). Happy to answer any questions here or if you prefer, via email: sean@wallaroolabs.com

I‘ve worked with Scala/Akka, and while I like the actors model, some things turn me off about the stack, mostly the language. After reading the Pony guiding principles, I was delighted. This looks like an implementation where the ecosystem complexities don’t get in my way of „getting things done“!

May I ask what turned you off about Scala?
Post reply on HN