Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built an auto-scalable Minecraft server for 1000+ players

worldql.com

Post reply on HN
Print view View original

131–140 of 145 posts

Re: How we built an auto-scalable Minecraft server for 1000+ players

#131

Post by rcxdude »

Thaxll wrote: post #29
ScaleneTriangle wrote: post #24

Earlier quoted context omitted.

If you look at the developer blogs for EVE Online, you'll have endless reading about massive scale multiplayer servers. Possibly the most technically impressive massively multiplayer experience

EvE is one of the worst architecture for an MMO, there is almost nothing to do and yet they need to slow down the simulation loop when there are too many players. I mean when you gameserver is based on Python what else can you do?

I think despite an unfortunate underlying architecture (which has especially bitten them in terms of development speed), they've achieved an impressive density. No other game I know of supports as many other players 'on screen' as EVE. The number of players you can have in a single fight is similar to the max number of online players on the larger WoW realmns

Re: How we built an auto-scalable Minecraft server for 1000+ players

#132

Post by steve-chavez »

Jaxkr wrote: post #59
xupybd wrote: post #52

Earlier quoted context omitted.

Do you use postgis for spatial queries?

No. All spatial queries use WorldQL’s in-memory k-d tree implementation and are not used for permanent world alterations. For the Minecraft world catch-up example, it’s as simple as querying Postgres for block records in a certain chunk after a certain timestamp. No fancy spatial stuff happens on the Postgres side.

Could yo elaborate on why you don't use PostGIS? I've always assumed its C algorithms are high-performance.

Re: How we built an auto-scalable Minecraft server for 1000+ players

#133

Post by Natfan »

tdeck wrote: post #40

I'm always amazed by how so much Minecraft innovation is driven entirely by hobbyists. For years Minecraft was unplayable on many machines without the Optifine plugin. Even with a huge corporation behind it, Microsoft didn't build the capacity for large-scale servers, and the very large paid servers seem to use region based sharding with portals. This project is truly exciting.

Minecraft servers aren't really great at region-based sharding.

Mojang have not yet added a "transfer packet"[0] which would allow for region-based switching.

For the most part, one would need to re-join the server under a different proxy pool located in their desired region, usually accessible via a subdomain (us.example.com, eu.example.com)

[0]: https://hypixel.net/threads/why-do-we-need-transfer-packets....

Re: How we built an auto-scalable Minecraft server for 1000+ players

#134

Post by tekknik »

stjo wrote: post #104

A couple of weeks ago I made https://playmcnow.com with the opposite idea - have many super lightweight servers for small groups of players. Instead of saying to friends “register to digital ocean, pay $5 and get a Minecraft box”, I can host thousands of worlds for them. I made a custom minecraft proxy (similar to bungeecord but A LOT less resource intensive) that starts the real server when someone attempts to conne…

How is this different than realms? I think realms is even cheaper in some cases.

https://www.minecraft.net/en-us/realms-for-java

Re: How we built an auto-scalable Minecraft server for 1000+ players

#135

Post by Kinrany »

stjo wrote: post #104

A couple of weeks ago I made https://playmcnow.com with the opposite idea - have many super lightweight servers for small groups of players. Instead of saying to friends “register to digital ocean, pay $5 and get a Minecraft box”, I can host thousands of worlds for them. I made a custom minecraft proxy (similar to bungeecord but A LOT less resource intensive) that starts the real server when someone attempts to conne…

Wonder if it can be pushed further: have more servers than players, with different mods but with the ability to move items (and players) between servers as long as they're compatible.

Re: How we built an auto-scalable Minecraft server for 1000+ players

#136

Post by bluedino »

jaflo wrote: post #98
e12e wrote: post #31

Earlier quoted context omitted.

It might be too pricy, but if you go to about 25usd/month, you can get a dedicated server with an i7 from hetzner server auction. These cheap boxes don't have ecc ram etc - but should work well for things like this. https://www.hetzner.com/sb?country=us

On the page, one option advertised is for €23.53/month with an Intel Core i7-4770, 2 TB HDD, and 32 GB RAM. What’s the catch? How is it possible to rent such hardware for so cheap?

Those are ancient servers, long since paid for, you're paying for 1.50 worth of bandwidth and basically some electricity and a little bit of "support"

Re: How we built an auto-scalable Minecraft server for 1000+ players

#137

Post by bluedino »

Jweb_Guru wrote: post #124
e12e wrote: post #122

Earlier quoted context omitted.

Nothing wrong with getting a used pc, but do you have 1gb uplink at home? In the cheapest offers right now, there's a Intel Core i7-4770, 2x 2 TB Ent. HDD (spinning rust, not ssd) with 32 GB ram. And a xeon box with ecc ram (same, low price).

Also, even if you do have a 1 Gbps uplink, a lot of providers will throttle you if they suspect you're actually hosting a server that tries to utilize most of that bandwidth in anything but bursts.

For example, 1TB monthly is really only 10mbs...common quota for hosted servers.

Re: How we built an auto-scalable Minecraft server for 1000+ players

#138

Post by evgen »

Animats wrote: post #79
forrestthewoods wrote: post #28

Earlier quoted context omitted.

Pretty much no video game engine scales with cores. There are too many dependent systems. There's no game engine that I'm aware of that effectively uses more than a couple of cores for gameplay simulation. Distributed physics is a well understood problem with no "solution". Just different trade-offs. This solution is cool, but it's not novel or anything. Minecraft is particularly challenging because the entire world…

There are no general solutions to any of this. Just a bunch of custom one-offs. SpatialOS is trying. My opinion on it is extremely, extremely negative. And I'll leave it at that. I'd like to hear more about that. nagle at animats.com, if you don't want to say much in public. This ties into the "metaverse" business. Lots of metaverse talk about big seamless worlds full of user-created content, but not much is running.…

Forgive the late follow-up, but my first startup in the valley did this back in the mid-90s (Electric Communities, everal EC founders and key hires are mentioned in the Snow Crash acknowledgements) and it is amusing to see people continue to make the same mistakes and step on the same landmines every few years. EC created an entire language (E) to try to hide the complexity and remain secure, had a PERT chart where an actual color on the chart (purple I think) was used to signify that this task would be a good PhD thesis for someone, and really only managed to get a feel for just how large this elephant was before eventually pivoting in a direction that had the possibility to lead to profit in the pre-dotCom era. At best, I think they managed to stumble in the right direction for close to a decade by collecting a large group of really smart people and somehow keeping them happy and engaged for several years and through the ability of the president of the company to get Hollywood people to keep throwing money at them year after year. If you want an interesting read for the evening you should find a copy of Chip Morningstar's 'Cyberspace Protocols Requirements' paper; it holds up incredibly well for something written almost 25 years ago when most people were still on dial-up and we thought 64MB of RAM and a Pentium Pro was too much to require on the client side...

Re: How we built an auto-scalable Minecraft server for 1000+ players

#139

Post by imtringued »

forrestthewoods wrote: post #28
Fiahil wrote: post #20

Earlier quoted context omitted.

Is there a specific reason why Minecraft hasn’t been “adapted” from the original game to a robust design that could scale to larger worlds and player populations before your project ?

Pretty much no video game engine scales with cores. There are too many dependent systems. There's no game engine that I'm aware of that effectively uses more than a couple of cores for gameplay simulation. Distributed physics is a well understood problem with no "solution". Just different trade-offs. This solution is cool, but it's not novel or anything. Minecraft is particularly challenging because the entire world…

I agree it's hard. It's only worth doing if your business depends on it.

Re: How we built an auto-scalable Minecraft server for 1000+ players

#140

Post by imtringued »

Jweb_Guru wrote: post #96
forrestthewoods wrote: post #73

Earlier quoted context omitted.

The original question was: > Is there a specific reason why Minecraft hasn’t been “adapted” from the original game to a robust design that could scale to larger worlds and player populations before your project ? The answer is because it's a lot of hard work. I am happy that a bunch of smart and talented people are working really hard to optimize Veloren. Good for you. I hope you help push the state of the art.

We have plenty of contributors who have never even programmed before, and certainly have no experience with parallel programming, and the bulk of our physics parallelization ( easily the trickiest part to make concurrent) has been done by people who aren't professional programmers. And I don't think I can recall a single change (proposed or implemented) by any of these contributors that resulted in a significant redu…

I apologize for the late response but here is a challenge to you.

Try to to parallelize a simplified form of applied energistics.

Applied energistics is a mod that lets you create an item transportation network. There are storage containers and machines with an inventory (for the sake of simplicity make them hold exactly 1 item and let the machines just turn A into B, B into C, C into A). The network interacts with inventories through interfaces. A storage interface makes items in that inventory accessible to every machine. Machines receive inputs through exporter interfaces and send outputs through importer interfaces.

It effectively is a database for items and that is exactly what makes it difficult to parallelize. The vast majority of games have 1:1 interactions between entities. In this system interactions can be as bad as n:m. That's also why it lags so badly with large networks. A hundred machines periodically scan hundreds of inventories.

Post reply on HN