141–150 of 179 posts
Earlier quoted context omitted.
6?!? We had only 4 colors in low res mode and 2 in high res
Well my Hercules graphics card was only monochrome, but it was relatively high resolution.
But there wasn't any similar programmatic debugging aid for detecting uninitialized stack memory.
Going further down the rabbit hole, I discovered the _chkstk function.
The MS C compiler would emit a call to _chkstk on function entry to ensure that stack memory had been paged in. But further reading noted that _chkstk was only emitted if the function allocated a lot of stack memory. And there was source code! MS included the assembly language source code for _chkstk in the CRT source code, installed with compiler.
I needed _chkstk to be emitted for every function not only for functions that allocated >= 4KB of stack variables.
Curses, foiled again.
Then, while perusing the list of compiler command line switches, I see "/Ge".
/Ge (Enable Stack Probes)
Activates stack probes for every function call that requires storage for local variables.
I had all the pieces I needed to fill uninitialized stack memory with a non-zero canary value so I could make detection of uninitialized stack variables more reliable.
_stkfil was born
Modifying _chkstk was easy. I needed to write to every byte of stack in a stack page instead of reading only 4 bytes and skipping to the next page of stack.
While I was mucking in the bowels of modifying _chkstk, I added a 4-byte global variable to hold my canary value. Let the app override what value to use.
In debug builds, _stkfil helped find a couple of bugs, but soon all the stray uninited stack vars were gone and the code was forgotten.
Then I read about InitAll in https://www.microsoft.com/en-us/msrc/blog/2020/05/solving-un...
InitAll - Automatic Initialization
In addition to the previously mentioned approaches, Microsoft is now using a feature known as InitAll which performs automatic compile-time initialization of stack variables.
This section documents how Windows is using this technology and the rationale for why.
Current Windows Settings
The following types are automatically initialized:
- Scalars (arrays, pointers, floats)
- Arrays of pointers
- Structures (plain-old-data structures)
The following are not automatically initialized:
- Volatile variables
- Arrays of anything other than pointers (i.e. array of int, array of structures, etc.)
- Classes that are not plain-old-data
For optimized retail builds, the fill pattern is zero. For floats the fill pattern is 0.0.
For CHK builds or developer builds (i.e. unoptimized retail builds), the fill pattern is 0xE2. For floats the fill pattern is 1.0.> Anyway, my colleague found that there was one program that needed to allocate around 64KB of memory on the stack and initialize it. The standard way of doing this is to perform a stack probe to ensure that 64KB of memory is available, then subtracting 65536 from the stack pointer, and then initializing the memory in a small, tight loop. Actually, the standard way of allocating 64 kB of memory on the stack is to jus…
IIRC you have to probe every page of the stack on Windows. You cannot just subtract a value from ESP/RSP. If you don't probe every page in order, you get a page fault or some other exception (I don't remember which one).
Earlier quoted context omitted.
I used to be a graphics card/chip architect for macs in the early/mid 90s - our chips were the fastest, but some programs were resistant because they did stupid stuff: pagemaker invalidated the font cache every time it went thru its main loop, quark with ATM did an n*2 thing every time it wrote text etc etc. We had special hardware to accelerate text drawing and it did nothing because the software pissed it away. We…
What would have been the purpose of stupid code like that? Was it a workaround for things that didn’t fully complete on one iteration, so the devs kept hammering away at it until it worked?
If the stream is buffered, then all operations, including fread, are supposed to go through the buffer.
All three of these should issue buffer-sized reads to the operating system:
1. A loop which calls getc(stream) 65536 times.
2. fread(buf, 1, 65536, stream)
3. fread(buf, 65536, 1, stream)
The more direct behavior of fread should only kick in if the stream is configured as unbuffered.
I would say that the way low-level reads are issued to the host operating system is a "visible effect" of the program, so I suspect this may actually be a matter of conformance. I.e. it's not okay to issue those reads however the stream library wants as long as the data is read.
Earlier quoted context omitted.
I used to be a graphics card/chip architect for macs in the early/mid 90s - our chips were the fastest, but some programs were resistant because they did stupid stuff: pagemaker invalidated the font cache every time it went thru its main loop, quark with ATM did an n*2 thing every time it wrote text etc etc. We had special hardware to accelerate text drawing and it did nothing because the software pissed it away. We…
I remember when 24 bit color was exotic and aspirational and you had to settle for 16.
8 bit psuedo color, so the color palette switched with every focus-follows-mouse window boundary crossing. 16 bit direct color with banding but no more palette psychedlia.
This was equal parts to make it faster and to allow for higher framebuffer resolutions with limited VRAM.
I was looking through the compiler docs about memory allocation and I found the section about the debug version of the CRT which could fill the allocated memory with a non-zero canary value to help detect uninitialized memory (assuming you weren't calling calloc - which zero-init's allocated memory). But there wasn't any similar programmatic debugging aid for detecting uninitialized stack memory. Going further down t…
However as someone who looks a lot at instruction traces I could probably write on e on why Linux kernel code sucks too. One of my current pet peeves is the way Linux walks bitmasks of CPU bits, which is a reasonably common operation. Due to a chain of unfortunate changes and decisions it currently needs 16+ instructions to find the next bit for something which the x86 instruction set has a single instruction. Of course that is so big that it is even outlined, adding even more overhead.
This reminds me of a story from 15 years ago, where I was developing a technology to download games on demand by hooking into the OS calls. There was a particular game that was superslow when this tech was applied. Original game loading took around 15-20 seconds, whereas once the tech was applied it took easily 3-5 min, even with all data already downloaded. When I started digging into it, I realized the reason was t…
That's because the OS does the same thing too. It's the right fix, when I implemented something similar, we implemented caching right away.
Earlier quoted context omitted.
This is not quite an accurate telling of rockstar's reaction, there were actually receptive to it and paid out $10k for the discovery. Though it's an understandable mistake given rockstar's hostile history with the gta modding scene. See the original post and discussion for the whole story: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times... https://news.ycombinator.com/item?id=26296339
That's not how I remember these events when they were playing out. I distincly remember social media posts warning about the dangers of modifying game files, plus refusal to acknowledge the issue. Note there were 2 full weeks between the blog post and the update mentioning the bounty. I'm pretty sure the massive community outrage in between has played a role in it. But I don't have any sources and I was wrong about a…