Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Akka Cluster Works: Actors Living in a Cluster

lightbend.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: How Akka Cluster Works: Actors Living in a Cluster

#51

Post by cutemonster »

hugofirth wrote: post #45

To offer a slightly dissenting opinion, we’ve had many issues with Akka over the years: - if you roll your cluster membership a lot the dotted version vectors which are created by Akka distributed data grow unbounded. Eventually they will start making gossip messages exceed the default maximum size (a few kB IIRC) and fail to send. - in the presence of heavy GC Akka cluster has a really bad time. Members will flip fl…

How would you compare Akka with Erlang? Or is that a weird thing to ask here

What made you choose Akka

Re: How Akka Cluster Works: Actors Living in a Cluster

#52

Post by alexis2b »

t-writescode wrote: post #48
valenterry wrote: post #38

Earlier quoted context omitted.

I don't understand what you mean. In my scenario, multiple machines are used and necessary for the same service (otherwise there is no point in using Akka Cluster). Example: online games. You have room/game being created on the fly and it is destroyed an hour later. There are many of these rooms and they are distributed over multiple machines. You can't really use rabbitMQ here, it's not performant enough. And even i…

You're not just talking online games, you're talking enormous-world MMOs, and that's such a far cry from the areas that I have worked with and thought about how to work with in any detail that I can't usefully add anything. If you're trying to manage the concurrent state of 10s of thousands of players in a game, all server-side, and you want specific actors to handle that single player and you don't want a globally p…

Interesting! Not trying to divert from Akka which is a wonderful piece of engineering, but your comment reminded me of Microsoft’s take at the Actor model with Project Orléans - which was used for the backend side of the Halo / XBox MMO [0].

I think the GA version of Project Orléans is now called Service Fabric, although I never had the pleasure to try it.

[0] https://youtu.be/I91ZU8tEJkU

Re: How Akka Cluster Works: Actors Living in a Cluster

#53

Post by valenterry »

t-writescode wrote: post #48
valenterry wrote: post #38

Earlier quoted context omitted.

I don't understand what you mean. In my scenario, multiple machines are used and necessary for the same service (otherwise there is no point in using Akka Cluster). Example: online games. You have room/game being created on the fly and it is destroyed an hour later. There are many of these rooms and they are distributed over multiple machines. You can't really use rabbitMQ here, it's not performant enough. And even i…

You're not just talking online games, you're talking enormous-world MMOs, and that's such a far cry from the areas that I have worked with and thought about how to work with in any detail that I can't usefully add anything. If you're trying to manage the concurrent state of 10s of thousands of players in a game, all server-side, and you want specific actors to handle that single player and you don't want a globally p…

> You're not just talking online games, you're talking enormous-world MMOs

I am talking about bigger scala here of course. But not necessarily what you describe.

Take games like League of Legends or Counterstrike as examples. Having one game per actor seems like a sensible design to me.

But yeah, I think that traditional techniques still get you very far. I heard that Slack was running just on (multiple) postgres for the longest time.

Re: How Akka Cluster Works: Actors Living in a Cluster

#54

Post by valenterry »

vinay_ys wrote: post #47
valenterry wrote: post #38

Earlier quoted context omitted.

I don't understand what you mean. In my scenario, multiple machines are used and necessary for the same service (otherwise there is no point in using Akka Cluster). Example: online games. You have room/game being created on the fly and it is destroyed an hour later. There are many of these rooms and they are distributed over multiple machines. You can't really use rabbitMQ here, it's not performant enough. And even i…

If you want to host a game state object in memory (vs serialized and saved to SSD) because you have lots of frequent write/reads happening to that object in a very short window of time such that the IO cost and CPU cost (ser/deserialisation) is higher and the incremental latency is a blocker, then this design of hosting a full-blown object in memory within your runtime makes sense (number of reads per game object per…

I mean, you are right that Akka Cluster is JVM based and hence can bring the problems you mentioned. But then again, most high frequency trading also runs on the JVM, so it can often be worked around.

> If this happens frequently enough (to affect say 5% of your users – enough to kill your game experience), is the benefits of latency gain from in-memory object reads wiped out?

For this specific use-case, I don't think there is really an alternative, except for specifically a hand-crafted system (or non-scale, such as everyone hosts and manages their own server).

Re: How Akka Cluster Works: Actors Living in a Cluster

#55

Post by hugofirth »

cutemonster wrote: post #51
hugofirth wrote: post #45

To offer a slightly dissenting opinion, we’ve had many issues with Akka over the years: - if you roll your cluster membership a lot the dotted version vectors which are created by Akka distributed data grow unbounded. Eventually they will start making gossip messages exceed the default maximum size (a few kB IIRC) and fail to send. - in the presence of heavy GC Akka cluster has a really bad time. Members will flip fl…

How would you compare Akka with Erlang? Or is that a weird thing to ask here What made you choose Akka

We're a big Java/Scala shop already, so Akka was easier to integrate :)

Re: How Akka Cluster Works: Actors Living in a Cluster

#56

Post by vinay_ys »

valenterry wrote: post #54
vinay_ys wrote: post #47

Earlier quoted context omitted.

If you want to host a game state object in memory (vs serialized and saved to SSD) because you have lots of frequent write/reads happening to that object in a very short window of time such that the IO cost and CPU cost (ser/deserialisation) is higher and the incremental latency is a blocker, then this design of hosting a full-blown object in memory within your runtime makes sense (number of reads per game object per…

I mean, you are right that Akka Cluster is JVM based and hence can bring the problems you mentioned. But then again, most high frequency trading also runs on the JVM, so it can often be worked around. > If this happens frequently enough (to affect say 5% of your users – enough to kill your game experience), is the benefits of latency gain from in-memory object reads wiped out? For this specific use-case, I don't thin…

> But then again, most high frequency trading also runs on the JVM, so it can often be worked around.

JVM is an awesome piece of technology. And you can do robotic control systems to high-frequency trading systems with it with careful programming.

But I've seen a lot of Java code running in production suffering from latency jitters and needing continuous profiling and optimization by a small group of performance engineers while the majority of application engineers keep adding to GC load.

> For this specific use-case, I don't think there is really an alternative, except for specifically a hand-crafted system

Yes, but I think the handcrafted system doesn't need to be very complex. It can be quite simple and easy to understand and tame to your needs as your scale and complexity grows.

Re: How Akka Cluster Works: Actors Living in a Cluster

#57

Post by AzzieElbab »

darksaints wrote: post #31
tunesmith wrote: post #2

In scala-land, a lot of people like to scoff at Akka because they prefer other pure fp concepts, but I don't think they've found a replacement for Akka Cluster - where you need objects that have both state and behavior, meaning they need to exist in memory, and where there are too many to exist on one server. If you don't need behavior, you can use things like distributed databases or caches, and if you don't need to…

I don't think the scala crowd scoffs at Akka because they are beholden to pure functional programming. The pure FP crowd is actually a minority...significant enough to acknowledge, but not enough to make or break anything about the community. The real problem with Akka is that, at least until very recently, you had to abandon any semblance of type safety if you used it. That was very frustrating to work with. I can t…

I wouldn’t say all type safety. It just couldn’t prevent you from sending unhandled messages to actors

Re: How Akka Cluster Works: Actors Living in a Cluster

#58

Post by alextheparrot »

thelittlenag wrote: post #28
alextheparrot wrote: post #22

Earlier quoted context omitted.

I wonder what it would look like trying to implement something like the IO monad backed by Akka actors. Can't recall anything off the top of my head that would make that untenable aside from the aforementioned scoffing.

Take a look at ZIO Actors ( https://zio.github.io/zio-actors/ ).

Thanks for the pointer, wasn't aware ZIO had an actor implementation. My interest was the inversion, though, which is “Could you make a (more or less) drop-in replacement for Future/IO/Task/ZIO that implements not just concurrency, but distributed concurrency using actors” (Where the actors could be Akka, ZIO, etc.).

Re: How Akka Cluster Works: Actors Living in a Cluster

#59

Post by AzzieElbab »

alextheparrot wrote: post #22
tunesmith wrote: post #2

In scala-land, a lot of people like to scoff at Akka because they prefer other pure fp concepts, but I don't think they've found a replacement for Akka Cluster - where you need objects that have both state and behavior, meaning they need to exist in memory, and where there are too many to exist on one server. If you don't need behavior, you can use things like distributed databases or caches, and if you don't need to…

I wonder what it would look like trying to implement something like the IO monad backed by Akka actors. Can't recall anything off the top of my head that would make that untenable aside from the aforementioned scoffing.

I have seen several such implementations. They are fairly robust although feel a bit awkward when compared to streams. Scala Fs2 and zstreams are just too good once you figure them out. Neither provide clustering of course

Re: How Akka Cluster Works: Actors Living in a Cluster

#60

Post by playing_colours »

vinay_ys wrote: post #46
playing_colours wrote: post #39

Earlier quoted context omitted.

I remember using Akka in the domain of IoT. A persistent actor represented a sensor: state and history of readings. There was a great feature in Akka Persistence: if an actor went idle for a while - no new sensor data, it would be offloaded from memory to the storage (Cassandra). As soon as the sensor started to send new signals again or the sensor state was queried by a user the actor got loaded back to memory.

How much sensor data is kept in memory in the actor object? What's the cost tradeoff between memory vs an SSD? I wonder if SSD based solution would still be cheaper and more scalable than live memory objects based solution.

What sensor data and how much depend on the access needs: you can keep N latest records or some rolling aggregated data, if they are frequently accessed by some rule engine. It's certainly not feasible to hold big chunks of sensor data in memory.
Post reply on HN