Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The time the x86 emulator team found code so bad they fixed it during emulation

devblogs.microsoft.com

Post reply on HN
Print view View original

121–130 of 179 posts

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#121

Post by Xirdus »

psanchez wrote: post #20

This reminds me of a story from 15 years ago, where I was developing a technology to download games on demand by hooking into the OS calls. There was a particular game that was superslow when this tech was applied. Original game loading took around 15-20 seconds, whereas once the tech was applied it took easily 3-5 min, even with all data already downloaded. When I started digging into it, I realized the reason was t…

Reminds me of the "community patch" to GTA Online from a few years ago. The game was plagued by 10+ minute loading times. The situation remained for years and only got worse with time. Some hacker figured out that the game spent 80% of loading time reading the in-game store listing file. The file was tens of megabytes IIRC, and it literally used the Schlemiel the Painter's Algorithm - for each entry, start reading from the beginning byte after byte. The hacker made a tiny patch that made it remember where it found the last entry. This cut the total loading time by 80%, from over 10 minutes to less than 3.

Edit: removed incorrect information.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#122

Post by Xirdus »

urbandw311er wrote: post #49
Taniwha wrote: post #26

Earlier quoted context omitted.

I used to be a graphics card/chip architect for macs in the early/mid 90s - our chips were the fastest, but some programs were resistant because they did stupid stuff: pagemaker invalidated the font cache every time it went thru its main loop, quark with ATM did an n*2 thing every time it wrote text etc etc. We had special hardware to accelerate text drawing and it did nothing because the software pissed it away. We…

This is a horrible and yet not unexpected insight into the internals of Excel

There's a good chance it was Excel's workaround for some other GPU's buggy behavior.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#123

Post by layer8 »

trelbutate wrote: post #72
urbandw311er wrote: post #49

Earlier quoted context omitted.

This is a horrible and yet not unexpected insight into the internals of Excel

> To be fair excel would erase places white that it wanted to write up to 9 times before it drew any black pixels I feel like I'm having a stroke trying to read this, what does it mean??

It’s necessary for erasing cat pixels.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#124

Post by tom_ »

Someone wrote: post #54
psanchez wrote: post #20

This reminds me of a story from 15 years ago, where I was developing a technology to download games on demand by hooking into the OS calls. There was a particular game that was superslow when this tech was applied. Original game loading took around 15-20 seconds, whereas once the tech was applied it took easily 3-5 min, even with all data already downloaded. When I started digging into it, I realized the reason was t…

> Which basically expanded back in the day to 65k reads of 1 byte for several MB file. Each fread translated to 65k reads of ReadFile Windows API What software did that that badly? If the code asks for (up to) 65,536 single byte items, why would you split that into 65,536 calls? Also, that change changes behavior. The old call could read anything from zero to 65,536 bytes, the new one only can read zero or 65,536 byt…

The standard says that fread calls fgetc multiple times for each object:

> For each object, size calls are made to the fgetc function and the results stored, in the order read, in an array of unsigned char exactly overlaying the object

(wording unchanged since C99)

If the file is unbuffered, depending on how the implementation handles buffering, and how it interprets the standard, then perhaps it does end up hitting a path where there's 1 ReadFile call per byte...

I don't know how most implementations get around this. Presumably it's valid to interpret "calls are made" as "behaving as if calls are made", meaning fread can copy data out of the FILE's buffer directly, or make calls directly to whatever routine fgetc defers to, rather than calling fgetc N times literally. Looks like glibc's fread does this.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#125

Post by HansHamster »

Taniwha wrote: post #100
trelbutate wrote: post #72

Earlier quoted context omitted.

> To be fair excel would erase places white that it wanted to write up to 9 times before it drew any black pixels I feel like I'm having a stroke trying to read this, what does it mean??

Well all they needed to do was erase the screen with white and draw on it, but their app's internal logic meant that they erased it more than once. I was capturing QuickDraw library calls - the low level graphics primitives, to figure out where the graphics time in apps was going and found out sometimes excel did it 9 times Of course users didn't see it more than once, but our hardware made all that wasted time run f…

Maybe their CRTs had horrible burn-in and they had to erase everything 9 times before it was gone...

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#126

Post by Asmod4n »

pbhjpbhj wrote: post #119
fsfod wrote: post #91

Earlier quoted context omitted.

Part of Windows Explorer actually does tons of tiny 4 byte ReadFile calls in to its tracking database like file when you delete a file. If you deleting lots of files this quickly adds up.

Is this why Windows takes so long to delete things?? Presumably those reads aren't done when using del from a console as that always seems a bit faster.

Windows Explorers zip implementation also seams to do 1 byte reads by the speed is has compared to every other zip implementation.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#127

Post by spauldo »

PaulHoule wrote: post #95
Taniwha wrote: post #26

Earlier quoted context omitted.

I used to be a graphics card/chip architect for macs in the early/mid 90s - our chips were the fastest, but some programs were resistant because they did stupid stuff: pagemaker invalidated the font cache every time it went thru its main loop, quark with ATM did an n*2 thing every time it wrote text etc etc. We had special hardware to accelerate text drawing and it did nothing because the software pissed it away. We…

I remember when 24 bit color was exotic and aspirational and you had to settle for 16.

I swear if Sun Microsystems was still around, their machines would still ship with 8-bit pseudocolor and you'd have to pay an extra $3k for 24-bit.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#128

Post by jonathanlydall »

pbhjpbhj wrote: post #119
fsfod wrote: post #91

Earlier quoted context omitted.

Part of Windows Explorer actually does tons of tiny 4 byte ReadFile calls in to its tracking database like file when you delete a file. If you deleting lots of files this quickly adds up.

Is this why Windows takes so long to delete things?? Presumably those reads aren't done when using del from a console as that always seems a bit faster.

Its slowness is also a function of security software or any other file system "filters" (I believe they're called) are installed.

For example, I run TortoiseGit which has a caching feature which is supposed to make it faster at showing what to commit. Disabling it increases the number of items I can delete per second in my Windows Explorer from about 1000 to about 3000 while making not making TortoiseGit operations meaningfully slower (that I can tell).

This is a Dev Drive [0] on my machine, it would probably be slower on my C: drive which has full Windows Defender real time file scanning.

[0]: https://learn.microsoft.com/windows/dev-drive/

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#129

Post by jonathanlydall »

Asmod4n wrote: post #126
pbhjpbhj wrote: post #119

Earlier quoted context omitted.

Is this why Windows takes so long to delete things?? Presumably those reads aren't done when using del from a console as that always seems a bit faster.

Windows Explorers zip implementation also seams to do 1 byte reads by the speed is has compared to every other zip implementation.

It is frustrating how slow .zip (and more recently .7z) support built into Windows Explorer is.

This is a great article on why it's so unreasonably slow to modify these archives: https://textslashplain.com/2021/06/02/leaky-abstractions/

But it doesn't seem to explain why it's so much slower at regular extraction.

Re: The time the x86 emulator team found code so bad they fixed it during emulation

#130

Post by exrook »

Xirdus wrote: post #121
psanchez wrote: post #20

This reminds me of a story from 15 years ago, where I was developing a technology to download games on demand by hooking into the OS calls. There was a particular game that was superslow when this tech was applied. Original game loading took around 15-20 seconds, whereas once the tech was applied it took easily 3-5 min, even with all data already downloaded. When I started digging into it, I realized the reason was t…

Reminds me of the "community patch" to GTA Online from a few years ago. The game was plagued by 10+ minute loading times. The situation remained for years and only got worse with time. Some hacker figured out that the game spent 80% of loading time reading the in-game store listing file. The file was tens of megabytes IIRC, and it literally used the Schlemiel the Painter's Algorithm - for each entry, start reading fr…

This is not quite an accurate telling of rockstar's reaction, there were actually receptive to it and paid out $10k for the discovery. Though it's an understandable mistake given rockstar's hostile history with the gta modding scene.

See the original post and discussion for the whole story:

https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times... https://news.ycombinator.com/item?id=26296339

Post reply on HN