71–80 of 160 posts
https://images.contentstack.io/v3/assets/bltad9188aa9a70543a...
The blog post states: "The existing network monitoring system measured the latency at the networking layer as shown by the green arrow."
As context this was a big international fiasco. Nobody was acting maliciously but fans of RNG and other teams felt wronged. RNG didn't want to attend due to covid restrictions on travel but Riot really wants Chinese viewers in international tournaments, hence all this online latency work. Pros on other teams were lied to by the false latency information which must be almost paranoia inducing to be told it's only 35ms…
You can in fact have that magical time-travel approach to get apparent latency lower than the real number. Keep a history of the game state at every tick, so you can roll back to any previous instant. If real latency from one client is 50 ms, and you're targeting 35 ms, you roll back the game state by 15 ms each time input comes in from that client, and re-run the game logic from that point forwards and send the resu…
I love the intent and actions behind this post. Finding an unfair bug mid-tournament is serious, and it takes courage and integrity on the part of the technical team to -listen to the players -investigate past what your own analytics are saying -find and fix the bug under time pressure -and disclose the (potentially embarrassing!) initial mistake I think the article itself could have been worded better though. It did…
The technical team acted honorably, but the upper stewardship at Riot games behaved horrendously. They sacrificed the competitive integrity of their sport because they didn't want to exclude RNG from the tournament, because they wanted the audience of the Chinese market. This article goes into more detail, giving a larger context for why this situation shouldn't have never happened in the first place. https://www.inv…
Or maybe because the tournament would've been shit without them? Who wants to watch an international tournament with only 1 of the clearly top 2 regions represented? WHole thing would've been a forgone conclusion at that point.
Earlier quoted context omitted.
You would use an external source of randomness, which you can definitely prove is sufficiently random.
You could do something like this: • Pre-publish, for each time batch, a public key. You could publish lists of these well in advance. • Let everyone submit, alongside each order, a number arbitrarily selected by them. It does not matter how they select the number, but it would be simpler if everyone chose distinct numbers. • When order processing is done, do it by the order of closeness of the submitted number to the…
Earlier quoted context omitted.
That sounds like a complex solution. Sometimes a dumb solution that works good enough is better than a complex solution that _probably_ can't be exploited.
It's complex on one end to reduce complexity on the other -- the trading companies wouldn't have to worry about millisecond optimizations if they trading batches were 200ms windows. So the wire lengths wouldn't matter but also not mattering is the processor, memory, software, etc. for the trading companies. Seems like a good tradeoff. And honestly the wire thing probably isn't real. Light moves 30cm in a nanosecond.…
it's part of the exchange's license
and they have several videos of the rather large wire
there's even a tom scott video on it: https://www.youtube.com/watch?v=d8BcCLLX4N4
They used a complex and buggy solution when a simple solution exists: Use the `tc` traffic control [0] program to configure the Linux kernel to add a fixed amount of latency to traffic from the local players [1]. If the game server does not run on Linux, they could put a Linux bridge/router in front of it. [0] https://en.wikipedia.org/wiki/Tc_%28Linux%29 [1] https://serverfault.com/a/841865
It looks like the error they made was this: They added too much latency because they forgot to account for the latency they added in the client. You can see from their architecture diagram (Figure 4) that the latency measurement didn't include the client delay. https://images.contentstack.io/v3/assets/bltad9188aa9a70543a... The blog post states: "The existing network monitoring system measured the latency at the netw…
Earlier quoted context omitted.
I never understood why they didn't use randomized length micro batches to solve this. Instead of processing orders instantly, wait between 200ms and 500ms and then process all orders that came in that window in random order. Then being 5ms closer to the server wouldn't matter.
I don't think I get how that solves the issue: you would have set a fixed cutoff time, where you switch from one window/batch to the next. It doesn't matter when you arrive within the window. But statistically, even for random window lengths, if you have a smaller latency, you will make the cutoff for the earlier window more often.
Reminds me of the cable lengths for black boxes connected to the network in Wall Street. Each cable is the same length regardless of which computer is closer to the access point.
I never understood why they didn't use randomized length micro batches to solve this. Instead of processing orders instantly, wait between 200ms and 500ms and then process all orders that came in that window in random order. Then being 5ms closer to the server wouldn't matter.
It looks like the error they made was this: They added too much latency because they forgot to account for the latency they added in the client. You can see from their architecture diagram (Figure 4) that the latency measurement didn't include the client delay. https://images.contentstack.io/v3/assets/bltad9188aa9a70543a... The blog post states: "The existing network monitoring system measured the latency at the netw…
I read that section as comparing what they logged before vs what the player experiences: the time taken for an input to be registered and it's effect communicated to the user. Maybe I'm wrong, but I don't think the intention was a precise description of the error, but rather the hazard of not having logs which reflect what the user experiences.
"The reason we did not find it sooner is that the cause of the issue was a code bug that miscalculated latency, which meant that the values in our logs were also wrong."
And from later in the post:
"Our logs were not displaying the issue because the calculation was wrong. It explained why the latency was worse in the venue than on the internet servers"
