Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I cut GTA Online loading times (2021)

nee.lv

Post reply on HN
Print view View original

11–20 of 221 posts

Re: I cut GTA Online loading times (2021)

#11

Post by ddtaylor »

flohofwoe wrote: post #3

Clickbait title ;) The problem wasn't strlen(), that just showed up at the bottom of the call stack when profiling. IIRC the actual problem was an exceptionally dumb JSON parser combined with parsing a pretty big JSON file (which probably also grew much bigger than the original developer working on that feature anticipated). Edit: the title has been edited in the meantime, originally it was something about strlen() b…

strlen() was a major component of the slowdown. Yes, the JSON file was large and strlen() was indirectly caused by sscanf() being used. But it's not accurate to say it's a clickbait title when a bulk of the slowdown was caused by constantly recalculating the length of the C string. The article shows in detail how he created a monkey-patched strlen() that cached the length to avoid recalculating it over and over which was attributed to the speedup.

The speedup gains were not caused by reducing the size of the JSON file or doing any exceptionally better JSON parsing. The speedups were a direct result of simply not calling strlen() over and over in an O(N^2) fashion.

Re: I cut GTA Online loading times (2021)

#13

Post by flohofwoe »

ddtaylor wrote: post #11
flohofwoe wrote: post #3

Clickbait title ;) The problem wasn't strlen(), that just showed up at the bottom of the call stack when profiling. IIRC the actual problem was an exceptionally dumb JSON parser combined with parsing a pretty big JSON file (which probably also grew much bigger than the original developer working on that feature anticipated). Edit: the title has been edited in the meantime, originally it was something about strlen() b…

strlen() was a major component of the slowdown. Yes, the JSON file was large and strlen() was indirectly caused by sscanf() being used. But it's not accurate to say it's a clickbait title when a bulk of the slowdown was caused by constantly recalculating the length of the C string. The article shows in detail how he created a monkey-patched strlen() that cached the length to avoid recalculating it over and over which…

All true, but replacing the strlen() with a caching version is just fixing the symptoms (of calling strlen too often... but the only realistic solution without the ability to recompile the game - so kudos to the author to come up with this hack).

I bet the "official" patch by Rockstar simply dropped in a less embarassing JSON parser library :)

Re: I cut GTA Online loading times (2021)

#14

Post by ddtaylor »

flohofwoe wrote: post #13
ddtaylor wrote: post #11

Earlier quoted context omitted.

strlen() was a major component of the slowdown. Yes, the JSON file was large and strlen() was indirectly caused by sscanf() being used. But it's not accurate to say it's a clickbait title when a bulk of the slowdown was caused by constantly recalculating the length of the C string. The article shows in detail how he created a monkey-patched strlen() that cached the length to avoid recalculating it over and over which…

All true, but replacing the strlen() with a caching version is just fixing the symptoms (of calling strlen too often... but the only realistic solution without the ability to recompile the game - so kudos to the author to come up with this hack). I bet the "official" patch by Rockstar simply dropped in a less embarassing JSON parser library :)

This article was written before Rockstar released a patch and the 70% speedup is from his strlen() caching.

Re: I cut GTA Online loading times (2021)

#15

Post by gbraad »

flohofwoe wrote: post #3

Clickbait title ;) The problem wasn't strlen(), that just showed up at the bottom of the call stack when profiling. IIRC the actual problem was an exceptionally dumb JSON parser combined with parsing a pretty big JSON file (which probably also grew much bigger than the original developer working on that feature anticipated). Edit: the title has been edited in the meantime, originally it was something about strlen() b…

The title is still not right as the poster is not the 'I' as in the title.

Re: I cut GTA Online loading times (2021)

#16

Post by flohofwoe »

ddtaylor wrote: post #14
flohofwoe wrote: post #13

Earlier quoted context omitted.

All true, but replacing the strlen() with a caching version is just fixing the symptoms (of calling strlen too often... but the only realistic solution without the ability to recompile the game - so kudos to the author to come up with this hack). I bet the "official" patch by Rockstar simply dropped in a less embarassing JSON parser library :)

This article was written before Rockstar released a patch and the 70% speedup is from his strlen() caching.

Yes I'm aware, that article basically triggered the official patch. Would be interesting to know if the Rockstar patch provided a further speedup though.

Re: I cut GTA Online loading times (2021)

#17

Post by whizzter »

flohofwoe wrote: post #13
ddtaylor wrote: post #11

Earlier quoted context omitted.

strlen() was a major component of the slowdown. Yes, the JSON file was large and strlen() was indirectly caused by sscanf() being used. But it's not accurate to say it's a clickbait title when a bulk of the slowdown was caused by constantly recalculating the length of the C string. The article shows in detail how he created a monkey-patched strlen() that cached the length to avoid recalculating it over and over which…

All true, but replacing the strlen() with a caching version is just fixing the symptoms (of calling strlen too often... but the only realistic solution without the ability to recompile the game - so kudos to the author to come up with this hack). I bet the "official" patch by Rockstar simply dropped in a less embarassing JSON parser library :)

You don't think that it's a bigger WTF that sscanf implementation calls strlen when parsing a number (when it iirc even ignores any non-digit characters for the actual number parsing so no reason it couldn't terminate that part w/o the extra call)?

Re: I cut GTA Online loading times (2021)

#19

Post by aardvarkr »

flohofwoe wrote: post #16
ddtaylor wrote: post #14

Earlier quoted context omitted.

This article was written before Rockstar released a patch and the 70% speedup is from his strlen() caching.

Yes I'm aware, that article basically triggered the official patch. Would be interesting to know if the Rockstar patch provided a further speedup though.

The author provided an update at the bottom. His hack cut the tone from 6m to 1m 30s. The official rockstar patch took it down to 1m 53s. It’s MUCH better but still off the mark by a bit.
Post reply on HN