Hi, author here! Some details on the model: * Trained 17GB of code from the top 10,000 most popular Debian packages. The source files were deduplicated using a process similar to the OpenWebText preprocessing (basically a locality-sensitive hash to detect near-duplicates). * I used the [Megatron-LM]( https://github.com/NVIDIA/Megatron-LM ) code for training. Training took about 1 month on 4x RTX8000 GPUs. * You can d…
If you want to look at some more fake samples, here are 256 generated snippets. These ones start at the beginning of the file each time, so you should get a better sense of context:
Why would you ever want to swap an uninitialized value into a buffer? You're wasting CPU cycles writing out data that you are guaranteed to never want to use. Why not just do a copy from the source buffer to the uninitialized one (as that is likely the half of the swap that is desired)?
I literally explained why I want to do that in my last paragraph?
Your last comment just talks about not detecting when you read uninitialized values, but obviously, you wouldn't read uninitialized values _if you never wrote them_?
Unless your use case is swapping with an uninitialized buffer to mark a buffer as "done" and detect further use of it?
I literally explained why I want to do that in my last paragraph?
Your last comment just talks about not detecting when you read uninitialized values, but obviously, you wouldn't read uninitialized values _if you never wrote them_? Unless your use case is swapping with an uninitialized buffer to mark a buffer as "done" and detect further use of it?
You're not understanding what I'm saying. I'm talking about what I see in the debugger when I'm debugging. When you see 0xCC in a variable in the debugger you know you probably had an out of bounds read. Because in debug mode the compiler and runtime leave these markers in uninitialized memory. For that to be helpful you need to swap until the max of the initialized sizes of both arrays, so that you preserve these markers. You defeat that helpful feature if you copy the uninitialized portion of the buffer instead of swapping.
Your last comment just talks about not detecting when you read uninitialized values, but obviously, you wouldn't read uninitialized values _if you never wrote them_? Unless your use case is swapping with an uninitialized buffer to mark a buffer as "done" and detect further use of it?
You're not understanding what I'm saying. I'm talking about what I see in the debugger when I'm debugging . When you see 0xCC in a variable in the debugger you know you probably had an out of bounds read. Because in debug mode the compiler and runtime leave these markers in uninitialized memory. For that to be helpful you need to swap until the max of the initialized sizes of both arrays, so that you preserve these m…
Oh I think I see, I never considered the scenario where the buffer was half uninitialized, I thought you meant you were swapping an (entirely) uninitialized buffer with an initialized one.
Hi, author here! Some details on the model: * Trained 17GB of code from the top 10,000 most popular Debian packages. The source files were deduplicated using a process similar to the OpenWebText preprocessing (basically a locality-sensitive hash to detect near-duplicates). * I used the [Megatron-LM]( https://github.com/NVIDIA/Megatron-LM ) code for training. Training took about 1 month on 4x RTX8000 GPUs. * You can d…
For those trying to guess, you can find the real code using the Debian code search service:
This looks like overfitting to me. Some of the GPT samples were definitely real code, or largely real code. One looked like something from Xorg, another like it was straight from the COLLADA SDK. It’s really hard to define what “truly new code” is, if it’s just the same code copy pasted in different order. Blah blah Ship of Theseus etc.
I'm 90% sure I just got a boost header which was apparently GPT-2 generated, hmmm. Sadly, I can't do back and see it again
I, one the other hand, got a section of code with a method that had a complex name and took a bunch of parameters but only ever returned true. I was sure it was auto generated... But no it was just bad (real) code.
You're not understanding what I'm saying. I'm talking about what I see in the debugger when I'm debugging . When you see 0xCC in a variable in the debugger you know you probably had an out of bounds read. Because in debug mode the compiler and runtime leave these markers in uninitialized memory. For that to be helpful you need to swap until the max of the initialized sizes of both arrays, so that you preserve these m…
Oh I think I see, I never considered the scenario where the buffer was half uninitialized, I thought you meant you were swapping an (entirely) uninitialized buffer with an initialized one.
Well that certainly could be a special case of the more general scenario I wrote; that's what would happen if it turned out the first buffer had nothing useful in it and the second one was full. But yeah, in general they'd both be partially filled: you have [0, ?, ?] and [1, 1, ?] and want to swap them. You wouldn't touch the last ? in either, but you would swap the [0, ?] with [1, 1], and in debug mode you'd see ? = 0xCC. Except the language doesn't really let you do that, even though fundamentally there should be nothing wrong with it, and in fact is likely to be desirable in practice.
I guessed GPT2 each time, 200 times in a row and only found that GPT2 was correct 89/200 times, so about 45% was GPT2 for me.
In [2]: scipy.stats.binom_test(89, 200, 0.5) Out[2]: 0.13736665086863936 Unusual to be this lopsided (1-in-7), but not crazy.
Reminder that the p-value of a test is NOT the probability of H0 being true, see [0]. It only shows that, if we assume a significance of 0.05 we cannot reject the hypothesis (in our case that 89/200 is the result of a binomial distribution with p=.5).