Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

prdeving.wordpress.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#41

Post by mtve »

whartung wrote: post #34

Speaking about WoW specifically, since I'm not familiar with the others, I've always been curious about their quest system. Specifically keeping track of what are available for the character efficiently along with the event system to flag quests as completed, etc. There's so many of them. I have to assume they're spatially limited. You enter a zone, or an area, and the system loads up all of the quests located in tha…

Sorry in advance for not answering right to your question, but you may check sources of ManGOS/TrinityCore/family WoW servers for that.

In short, from what I know, yes, quests are stored in "quest log" fields of character data in server DB, and they are tracked by the clients and checked by the server. Some simple auto quests like "find this item" are not even tracked by server and only stored on completion. Since both client and server have all game data, the client knows about all possible quests and only shows to the player what is appropriate at a current state.

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#42

Post by PartiallyTyped »

sleepybrett wrote: post #33
sshagent wrote: post #13

Earlier quoted context omitted.

azerothcore is probably the best and most polished, if you don't mind wotlk

the last expansion that was any good?

Depending on who you ask.

Tangent: Imho, the only reason it is good is because it's not as grindy and / or the community just didn't put as much emphasis on min-maxing things. GearScore was a thing of course, but theory crafting wasn't anywhere close to what we have now.

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#43

Post by genocidicbunny »

whartung wrote: post #34

Speaking about WoW specifically, since I'm not familiar with the others, I've always been curious about their quest system. Specifically keeping track of what are available for the character efficiently along with the event system to flag quests as completed, etc. There's so many of them. I have to assume they're spatially limited. You enter a zone, or an area, and the system loads up all of the quests located in tha…

> Specifically keeping track of what are available for the character efficiently along with the event system to flag quests as completed, etc.

This is not necessarily that difficult, at least the first part. A lot of games will have quests be given out by a 'quest giver' character of some sort, or they will activate at specific interaction points on the map. You can do some cheap 'has-player-finished-quest' type of checks to determine if for example the quest giver has some sort of UI to indicate they have a quest available that activate when the quest giver first comes into view range. Quests with more initial conditions can hide their checks behind the interaction with the quest giver.

Doing quest progression can be a bit more challenging. You need to determine when to do the checks for progress, and also how comprehensive you want them to be. The more complex the check, the less often you can run it without affecting game performance. I've seen designers use all sorts of tricks depending on the specific quest. Interaction volumes that run checks, periodic ticks, on entity flash messages..etc.

> Then, every event can practically just be brute forced across your pending quests to see which ones get progressed

This only works for games that have a small number of active quests and not a lot of events. And with MMO's, you really need to be considerate of the accidental quadratic performance problem.

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#44

Post by dmoy »

Stevvo wrote: post #39
justinlloyd wrote: post #29

The "cache, lots of cache" statement is the most true of any MMO architecture we can build. I did some optimization work earlier this year on a project where the single back-end server is now handling 2 billion requests per minute and had around 3TB of RAM for cache (I think the final production system was aiming for 12TB of RAM). There's concerns around race conditions as you pointed out, message passing from client…

How did they get the architecture so wrong on that "open-world game that involves driving cars and running pedestrians over"?

I assume that is a tongue-in-cheek description for Grand Theft Auto (whichever one they made online). That game series has had various moral discussions / controversy surrounding it since before the first game came out like 25 years ago.

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#45

Post by BryantD »

Eumenes wrote: post #5

Early MMOs (WoW, Asheron's Call, Everquest, Daoc) were very impressive in terms of distributed computing.

And the techniques in the article are basically how we did it. (I say “we” but I’m just a system administrator who got to work with some cool people at Turbine.)

As Eumenes notes, the Asheron’s Call engine was significantly ahead of its time with the seamless zoning. The cost was high, though — we needed quite a few servers to run a world, I think many more than our competitors. There are business reasons why we weren’t quite as cost-conscious as we perhaps should have been.

The other factor involved in determining how often you persist is item duplication. If it’s possible to transfer an item between players without persisting state, and if there are known exploits that crash servers (not world, but individual servers), you wind up with an exploit that can duplicate items. But I’m sure that’s just hypothetical.

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#46

Post by doctorpangloss »

justinlloyd wrote: post #29

The "cache, lots of cache" statement is the most true of any MMO architecture we can build. I did some optimization work earlier this year on a project where the single back-end server is now handling 2 billion requests per minute and had around 3TB of RAM for cache (I think the final production system was aiming for 12TB of RAM). There's concerns around race conditions as you pointed out, message passing from client…

Let's say that among developers there was a history of how-to-implement gameplay traditions (like how to implement third person player movement, "gameplay ability systems", etc.) in programming languages besides C++. Like C# & Java, the memory managed friendly ones with good tooling. And let's say you're forbidden from reinventing C++ inside C# or Java, like Unity's Burst does (so called HPC#). But you can "do ECS," there are C# and Java ECS frameworks, that even use those languages respective arena allocation techniques well. You just aren't allowed to reinvent C++, but you can use a high performance middleware that does, like Netty.

Would you choose to author an MMO backend in one of those friendlier ecosystems?

Do you think there's value in having access to other Java applications, to embed as libraries of your grander "in memory" ideas?

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#47

Post by anon56 »

dmoy wrote: post #44
Stevvo wrote: post #39

Earlier quoted context omitted.

How did they get the architecture so wrong on that "open-world game that involves driving cars and running pedestrians over"?

I assume that is a tongue-in-cheek description for Grand Theft Auto (whichever one they made online). That game series has had various moral discussions / controversy surrounding it since before the first game came out like 25 years ago.

Not Carmageddon?

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#48

Post by PsylentKnight »

dmoy wrote: post #44
Stevvo wrote: post #39

Earlier quoted context omitted.

How did they get the architecture so wrong on that "open-world game that involves driving cars and running pedestrians over"?

I assume that is a tongue-in-cheek description for Grand Theft Auto (whichever one they made online). That game series has had various moral discussions / controversy surrounding it since before the first game came out like 25 years ago.

I assume he got that and he was referring to the jankiness of GTA Online.

For example, it took 5+ minutes to load for years, until some random guy fixed it for them [1]. Though I suppose that had little to do with the overall architecture.

[1] https://news.ycombinator.com/item?id=31681515

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#49

Post by machiaweliczny »

justinlloyd wrote: post #36
mannyv wrote: post #31

In most data architectures the DB is only the backing store, because no matter how fast your database is it's going to be slower than RAM. Once you start caring about the performance the second thing you do is stick a cache layer of one sort or another in front of the database; the first thing should be making sure you have the correct indexes. In any case, it sounds like a distributed cache problem. I wonder if you…

You can use redis or memcached, but every MMO or online game I've been involved with, unless it was a "web game", has eschewed those for the most part. The game server maintains the state, knows all the objects in the universe, or at least its portion of the universe, and is responsible for retrieving and updating those objects. Even redis and memcached would be considered slow by comparison. Those game objects/world…

What protocols are used to stream updates to client? Or it’s simulation on client and state dump on fails?

Re: MMO Architecture: Source of truth, Dataflows, I/O bottlenecks and how to solve

#50

Post by Aerroon »

Thaxll wrote: post #11
quadral wrote: post #8

You missed the opportunity to talk about WoW private servers like Trinitycore. Trinitycore emulator can handle 10k+ players on a single server.

I 've seen those so called 10k players per server and in reallity it just does not work and it's pretty much a lag fest. You should see how those servers run in China ( where private wow server are very popular ).

Is it the server that's laggy or the client?
Post reply on HN