It Can Happen to You • Hacker News Board

Re: It Can Happen to You

#181

Post by scaramanga » Thu, Mar 04, 2021, 7:12 AM UTC

fsociety wrote: post #42

scaramanga wrote: post #35

"it will open a 97 MB binary STL file in about 165 milliseconds flat, on a 2013 Macbook Pro. This is blinding fast." This actually sounds incredibly slow, that's nearly 1/5th of an entire second. What can it possibly be doing? :) In case anyone else was wondering, I followed the link and clicked the description and this is actually based on the time to the first frame being rendered - not just the time to load the fi…

How can loading a file be infinitely fast? There is latency to receive the first byte from loading the file.

When there is nothing to do. Like with an array of fixed sized structures all you need to know is how many there are and then you can increment a pointer past any number of those objects, and that pointer increment itself can probably be compiled away to nothing.

It depends on exactly what you're measuring, you see. If you are loading data in order to do something, then the "do something" part will incur 100% of all the costs in that case. So when you subtract the "do something" part to just be left with the "load it" part, you can end up with a meaningless zero-to-do/doing-it-infinitely-fast kind of a result.

So then, what would you measure? Just peak RAM throughput? You can get that from the data-sheets :)

Re: It Can Happen to You

#182

Post by blackoil » Thu, Mar 04, 2021, 7:15 AM UTC

loeg wrote: post #170

progre wrote: post #156

Earlier quoted context omitted.

There is a standard, sure. But there are also a lot of compilers out there and I would bet that all but a few has either a "this compiles c11 except for [list of unimplemented features]" caveat or non-standard extensions.

> But there are also a lot of compilers out there and I would bet that all but a few has either a "this compiles c11 except for [list of unimplemented features]" caveat or non-standard extensions. Your statement is broadly reasonable. Here's the GP: > Shouldn’t we just come clean and admit to ourselves that there is no such thing as the C standard ? There is a collection of loosely related languages that look similar…

Don't know about C, but

In Java/C#/Python, you have a standard, a dominant implementation which is compliant to that standard. Few more independent/niche implementations which may not be 100% compliant. Do we have a similar implementation in C?

Re: It Can Happen to You

#183

Post by scaramanga » Thu, Mar 04, 2021, 7:19 AM UTC

titzer wrote: post #69

scaramanga wrote: post #35

"it will open a 97 MB binary STL file in about 165 milliseconds flat, on a 2013 Macbook Pro. This is blinding fast." This actually sounds incredibly slow, that's nearly 1/5th of an entire second. What can it possibly be doing? :) In case anyone else was wondering, I followed the link and clicked the description and this is actually based on the time to the first frame being rendered - not just the time to load the fi…

Do anything with a file where you actually have to touch every byte (e.g. parsing), it is pretty impressive to get anything to go faster than 100MB/s. 500MB/s is blindly fast, IMHO.

Yeah, parsing text with a state machine is slow. Parsing, say, HTTP at that speed would be impressive without SIMD. But this is a binary file full of fixed sized structures, hence my confusion.

Anyway, the answer is there, it's actually measuring the performance of sending the mesh to openGL, to the GPU, and getting a frame rendered.

Re: It Can Happen to You

#184

Post by thaumasiotes » Thu, Mar 04, 2021, 7:20 AM UTC

alexchamberlain wrote: post #166

0xcde4c3db wrote: post #120

Earlier quoted context omitted.

It's easy to forget that the original C standards were largely codifying existing practice during an era when using gets() [1] was existing practice. The world wasn't quite ready for Ada, I guess. Best-laid plans of mice and men etc. etc.. Also, keep an eye out for "amortized" complexity. This does have a legitimately rigorous definition, but for latency-bound paths it can practically amount to "O(whatever), except f…

It makes me chuckle when hash maps are stated to be O(1) insertions. Which is true, in respect to the number of items in the map, assuming the map doesn't need resizing and there isn't a hash collision... but it's generally not true in respect to the key length. (I think most implementations are O(ln), where l is the length of the key and n is the number of inserted items, assuming the hash function is O(l) - the _am…

> assuming the map doesn't need resizing

This isn't a big difficulty; it's still amortized O(1).

> and there isn't a hash collision

This is a real difficulty, unless you allow map resizing. Luckily, we do.

> but it's generally not true in respect to the key length.

OK, but in most cases the key length is constant, making anything that depends on the key length O(1) by definition.

Re: It Can Happen to You

#185

Post by scaramanga » Thu, Mar 04, 2021, 7:21 AM UTC

mkeeter wrote: post #40

scaramanga wrote: post #35

"it will open a 97 MB binary STL file in about 165 milliseconds flat, on a 2013 Macbook Pro. This is blinding fast." This actually sounds incredibly slow, that's nearly 1/5th of an entire second. What can it possibly be doing? :) In case anyone else was wondering, I followed the link and clicked the description and this is actually based on the time to the first frame being rendered - not just the time to load the fi…

Touché – after all, disk-to-RAM is hundreds of MB/s, and faster if it's cached! In practice, I'm racing mesh loading against "how long does the OS take to give you an OpenGL context", which is rarely below 160 ms (longer if it has to switch from integrated to discrete GPU).

Great data-point, I was wondering what the costs of all that would be.

How do you measure when the frame is done, are you waiting for a vsync?

Re: It Can Happen to You

#186

Post by namibj » Thu, Mar 04, 2021, 7:22 AM UTC

wruza wrote: post #122

mkl wrote: post #102

Earlier quoted context omitted.

I think your code would be pretty much the same, sscanf, strlen and all. The main differences would be the standard library's implementations of strlen and whatever function you use to read the file into a string in the first place.

str_t json = loadfile(); size_t offset = random(); sscanf(“%d”, ?); With opaque str_t you can’t just json[offset]. Should sscanf take offset with every string (sscanf(fmt, s, off))? Should we copy a slice of json and parse it? Should str_t have zerocopy mirroring ability (s2 = strmirror(s, off, len))? How many of these three are just a snakeoil that changes nothing? It’s only pretty much the same until you try to wri…

Rust's &str type, or the non-unicode-assuming version &[u8], allow creating (sub-)slices, which probably matches your strmirror function. Except that the same syntax works for all (dynamic/static) length arrays, and even allows custom code to e.g. transparently wrap an SoA[0] representation.

[0]: https://en.wikipedia.org/wiki/AoS_and_SoA

Re: It Can Happen to You

#187

Post by loeg » Thu, Mar 04, 2021, 7:25 AM UTC

blackoil wrote: post #182

loeg wrote: post #170

Earlier quoted context omitted.

> But there are also a lot of compilers out there and I would bet that all but a few has either a "this compiles c11 except for [list of unimplemented features]" caveat or non-standard extensions. Your statement is broadly reasonable. Here's the GP: > Shouldn’t we just come clean and admit to ourselves that there is no such thing as the C standard ? There is a collection of loosely related languages that look similar…

Don't know about C, but In Java/C#/Python, you have a standard, a dominant implementation which is compliant to that standard. Few more independent/niche implementations which may not be 100% compliant. Do we have a similar implementation in C?

I think you're maybe overstating the degree of homogeneity in Java/C#/Python. E.g., CPython3 cannot run Python2 code at all, and there are a lot of platform-specific modules and behaviors in the stdlib ("import os"). I don't think Python has any real single standard to the level of detail that C does, although I may be mistaken.

For C, on approximately the same platforms, to approximately the same degree: either GCC or Clang would be the corresponding standard-compliant implementation.

Re: It Can Happen to You

#188

Post by tpetry » Thu, Mar 04, 2021, 7:26 AM UTC

I don‘t get the heat of this topic. Yes they wrote some very slow code because it‘s easy to shoot in your foot with scanf. It‘s nothing new that most software could be heavily optimized by just benchmarking slow parts. There is no reason for this shit storm than to feel better than other developers.

The real problem is that they shipped a game with a loading screen which is taking minutes and not looking whether they could optimize it. THAT is the real shit show!

Re: It Can Happen to You

#189

Post by dorgo » Thu, Mar 04, 2021, 7:32 AM UTC

alexchamberlain wrote: post #166

0xcde4c3db wrote: post #120

Earlier quoted context omitted.

It's easy to forget that the original C standards were largely codifying existing practice during an era when using gets() [1] was existing practice. The world wasn't quite ready for Ada, I guess. Best-laid plans of mice and men etc. etc.. Also, keep an eye out for "amortized" complexity. This does have a legitimately rigorous definition, but for latency-bound paths it can practically amount to "O(whatever), except f…

It makes me chuckle when hash maps are stated to be O(1) insertions. Which is true, in respect to the number of items in the map, assuming the map doesn't need resizing and there isn't a hash collision... but it's generally not true in respect to the key length. (I think most implementations are O(ln), where l is the length of the key and n is the number of inserted items, assuming the hash function is O(l) - the _am…

But, isn't the key length a constant and we are back to O(1)? Ok, in theory you could exhaust all possible keys of a certain length and proceed with longer keys. It would give us what? O(ln(n))?

Re: It Can Happen to You

#190

Post by kazinator » Thu, Mar 04, 2021, 7:45 AM UTC

jchw wrote: post #85

kazinator wrote: post #18

Earlier quoted context omitted.

I see you are reiterating this point raised in the previous discussion several days ago, but I don't thing it is particularly well grounded. ISO C allows strtod and strtol to accept, other than in the "C" locale, additional "subject sequence forms". This does not affect programming language implementations which extract specific token patterns from an input stream, which either are, or are transformed into the portab…

I can't really understand what you mean. You can validate yourself that parsing with strtod will break if your system's locale is set to a locale where the decimal separator is a comma ',' instead of a period '.' - as an example, most European locales. Whether or not strtod will try to magically fall back to "C" locale behavior is irrelevant because it is ambiguous. For example, what do you do if you are in Germany a…

Sorry about that; I see the next now in the description of strtod where it is required that the datum is interpreted like a floating-point constant in C, except that instead of the period character, the decimal point character is recognized.

Yikes!

There is a way to fix that, other than popping into the "C" locale, which is to look up that decimal character in the current locale, and substitute it into the string that is being fed to strtod. That's a fairly unsavory piece of code to have to write.

What locale issues did you find for strol{l,ul,ull}?

> I really think it is just not a good idea to use the C standard library for string operations.

I really despise locale-dependent behavior also, and try to avoid it.

If you control the entire program, you can be sure nothing calls setlocale, but if you are writing middleware code, you can't assume anything.

Also in POSIX environments, the utilities react to localization environment variables; they do call setlocale. And so much stuff depends on it, like for instance operators in regex! [A-Z] does not refer to 26 letters; what [A-Z] denotes in a POSIX regex depends on the collation order of the character set.

There are functions in the C library without locale-specific behaviors, though, like strchr, strpbrk, strcpy, and their wide character counterparts.