Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring my Minecraft server with OpenTelemetry and Prometheus

dash0.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#51

Post by strogonoff »

mmanciop wrote: post #40
strogonoff wrote: post #28

Earlier quoted context omitted.

I agree, and I am as guilty of procrastination. However, the author is not really procrastinating—he gets paid for this. Me, I do in fact procrastinate on setting up a Minecraft server infra in the cloud. Maybe that’s precisely why the solution to this problem strikes me as inadequate: > So, the Minecraft server should work reliably and, if it goes down, I should know well before they do How are metrics helpful? Ther…

> How are metrics helpful? There is so much fun that could be had in setting up an actually resilient system instead. Metrics are the means to an end of alerting. And with alerting, I mean getting pinged on my phone when something important breaks. Like, you know, the server going down. > Why worry over metrics and alerts when you could orchestrate an infrastructure that grants you the superpower of being able to spi…

Do you really need to be urgently made aware that it’s down, if the system could simply spin up a new container and keep on as it were? You could still see that it had to do it, and if in the mood investigate it, but the matter of first importance is taken care for you.

> As somebody who has run cloud and enterprise software for almost two decades now, I can be that needs monitoring too

To be clear, I strongly believe that if you run anything seriously in production, you must monitor it—but first you need to be able to spin it back up with minimal effort. It may take a while to get there if you just inherited a rusty legacy snowflake monolith that no one dares to breathe the wrong way near, but if you are starting anew it is a bad mistake to not have that down first considering how straightforward it is nowadays.

Then, for hobby projects of low criticality (because people in this thread mistakenly assume I mean any personal project, I have to reiterate: nothing controlling points of ingress into your house or the like), you may find that once you have the latter, the former becomes optional and not really that interesting anymore.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#52

Post by strogonoff »

mmanciop wrote: post #39
strogonoff wrote: post #25

Earlier quoted context omitted.

And sometimes a person is paid to pretend toil is fun. We are talking about spending hours setting up telemetry instead of playing a game .

I swear I had a lot of fun setting doing the setup. I am also a massive observability nerd, so YMMV :-)

I believe you! Just due to your affiliation I wanted to highlight to any newbie SREs in the audience that perhaps there is a better way. I still think my approach is better, but we can do things differently.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#53

Post by strogonoff »

harrall wrote: post #31
strogonoff wrote: post #7

Monitoring and metric collection makes a lot of sense when you run a production system, or a personal but critical system. Promoting a telemetry solution when it comes to a hobby server, which you host for yourself and which can’t bankrupt you by running up a massive AWS bill, doesn’t seem to make much sense when simply bottling it up in Docker and being able to restart or recreate at will is enough (mount volumes fo…

Setting up telemetry is really easy if you’ve done it before and it’s a learning opportunity if you haven’t. I have Dockerfiles from 10 years ago for Grafana and a time-series DB so basically you learn it once and you can bang out basic telemetry infra in an hour afterwards. And I still actually use InfluxDB and Grafana for my hobby stuff. My current Dockerfiles just look like my old ones…

What happens if Grafana or InfluxDB is down? Who monitors the monitors?

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#54

Post by dewey »

strogonoff wrote: post #50
dewey wrote: post #29

Earlier quoted context omitted.

You are somehow very negative about this piece and are not understanding that your definition of fun is not universal. As you said "There is so much fun that could be had in setting up an actually resilient system instead.", maybe the author has more fun setting up alerts and metrics instead of a resilient system like you do? The truth is that in most real-world scenarios getting alerts, metrics is much more importan…

> The truth is that in most real-world scenarios getting alerts, metrics is much more important than building a fully resilient system (Expensive, maybe overengieering for early stage etc.). Funny, because I have the opposite opinion. Build for failure first; if it’s critical/production then also monitor, but if an earthquake takes down an EC2 zone and you have no ability to spin it up exactly the way it was then the…

This thread isn't going anywhere. If your startup hasn't found paying customers there's no need to build earthquake-resilient software. For most businesses that are not billion dollar companies there isn't.

Of course for engineers that's a nice challenge, but that's the reason why engineers without a business sense have a hard time building their own companies if you prioritize perfect code and overengineered infrastructure over talking to customers or building the business.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#55

Post by strogonoff »

mmanciop wrote: post #45
strogonoff wrote: post #14

Earlier quoted context omitted.

> Family game server going down can be very stressful, especially if you have kids. Telemetry does not address this, though. Shoving it into a container and assigning it a simple “restart if down” rule does. Minecraft is a flaky beast, if you run snapshots and/or mods. Metrics or not, often “start again” is all you need. Furthermore, this is a game that adds new gameplay features multiple times per month. If you do n…

> Telemetry does not address this, though. Shoving it into a container and assigning it a simple “restart if down” rule does. A Systemd unit as shown in [1] does it too without using containers and with fewer moving parts of using containers. I use containers every day at $work. I have been using containers since before Docker was a thing. In this case, it's entirely overkill: Systemd units use the important things l…

There are no official ones (Microsoft profits from operating its own servers, why would it make things any easier), but there are community-maintained images.

I found that vanilla server is insufficient and an ability to declaratively define mods, the seed, OP players, etc. through container environment is very important for iterative evolution, but of course this is individual.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#56

Post by strogonoff »

dewey wrote: post #54
strogonoff wrote: post #50

Earlier quoted context omitted.

> The truth is that in most real-world scenarios getting alerts, metrics is much more important than building a fully resilient system (Expensive, maybe overengieering for early stage etc.). Funny, because I have the opposite opinion. Build for failure first; if it’s critical/production then also monitor, but if an earthquake takes down an EC2 zone and you have no ability to spin it up exactly the way it was then the…

This thread isn't going anywhere. If your startup hasn't found paying customers there's no need to build earthquake-resilient software. For most businesses that are not billion dollar companies there isn't. Of course for engineers that's a nice challenge, but that's the reason why engineers without a business sense have a hard time building their own companies if you prioritize perfect code and overengineered infrast…

I don’t think running a container, which takes one command and one small YAML file, is either overengineering or difficult.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#57

Post by mmanciop »

strogonoff wrote: post #52
mmanciop wrote: post #39

Earlier quoted context omitted.

I swear I had a lot of fun setting doing the setup. I am also a massive observability nerd, so YMMV :-)

I believe you! Just due to your affiliation I wanted to highlight to any newbie SREs in the audience that perhaps there is a better way. I still think my approach is better, but we can do things differently.

Indeed if there were “official” container images out there, I might have instead run the server on Google Cloud Run or AWS AppRunner, without having to take care of the Linux underneath. Or an Amazon ECS task. I don’t have a Kubernetes cluster, but I will at some point make a version of this blog to run it on K8s :-)

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#58

Post by frollogaston »

treyd wrote: post #33
strogonoff wrote: post #27

Earlier quoted context omitted.

From my understanding, Paper and the like are good for Minecraft servers focused around specific mini-games (rather than freedorm building), and are the only sensible choice for servers with many people (or not that many people, but really underpowered hardware). However, they may be a problem if players are sensitive to possible non-vanilla behaviour (as you mentioned, and it’s not limited to cheaty duping). Thankfu…

It's kinda nuts. The upstream mojang server binary starts to groan if you have >4-5 players on the same server doing stuff. They've really been dropping the ball on optimization in recent years. Paper is good enough for anyone but very technical players pushing to the limits of redstone tick timing logic, entity behavior, chunk loading mechanics, etc. These don't matter even for advanced players doing normal things.

Wasn't it always like this? There's a lot going on in the game, especially if generating new chunks, and it's in Java.

Re: Monitoring my Minecraft server with OpenTelemetry and Prometheus

#60

Post by treyd »

frollogaston wrote: post #58
treyd wrote: post #33

Earlier quoted context omitted.

It's kinda nuts. The upstream mojang server binary starts to groan if you have >4-5 players on the same server doing stuff. They've really been dropping the ball on optimization in recent years. Paper is good enough for anyone but very technical players pushing to the limits of redstone tick timing logic, entity behavior, chunk loading mechanics, etc. These don't matter even for advanced players doing normal things.

Wasn't it always like this? There's a lot going on in the game, especially if generating new chunks, and it's in Java.

It was not always like this. You used to comfortably be able to handle 70+ players in a single server before Paper existed (my memory of this is from before like 2015). You'd need to allocate a lot more memory than normal, like 8 gigs instead of the normal suggestion of 1 or 2, but it could handle it without regular lag.
Post reply on HN