It Can Happen to You • Hacker News Board

Re: It Can Happen to You

#371

Post by dcolkitt » Thu, Mar 04, 2021, 6:19 PM UTC

specialist wrote: post #365

johnfn wrote: post #32

Loving the progression here. Tomorrow, someone’s going to reduce the boot times of macOS by 90% by the same principle. A week from now, someone will prove P=NP because all the problems we thought were NP were just running strlen() on the whole input.

You kid. But truer things are said in jest. > ...Tomorrow, someone’s going to reduce the boot times of macOS by 90% by the same principle. My 2019 MacBook often pauses when I connect the charging cable. Sometimes it just seizes, requiring a hard bounce. Clearly there's a contended lock buried deep. Something non-obvious. I'm certain everything these days has dozens of hidden quadratics and contended locks. Which is o…

> My 2019 MacBook often pauses when I connect the charging cable. Sometimes it just seizes, requiring a hard bounce.

Yesterday my MBP kernel panicked because my keyboard was low on battery and the bluetooth connection kept dropping. There's something weird with MacOS where peripherals seem to really not be well isolated from the core OS runtime.

Re: It Can Happen to You

#372

Post by marton78 » Thu, Mar 04, 2021, 6:21 PM UTC

TeMPOraL wrote: post #199

johnfn wrote: post #32

Loving the progression here. Tomorrow, someone’s going to reduce the boot times of macOS by 90% by the same principle. A week from now, someone will prove P=NP because all the problems we thought were NP were just running strlen() on the whole input.

You're joking, but now I'm thinking about the XML we parse at work and the library we're using to do it. We parse a lot of it, but I've always had this vague feeling that it takes a bit too long (given the codebase is C++). The XML library we use is rather well-known, so if someone found a bug like this there, I'd suspect a general improvement of performance across the board in the entire industry. Efficient Market H…

What about parsing that XML upfront, serialising to some binary format (e.g. CBOR, maybe with nlohmann's JSON library, or Cap'n Proto) and shipping the binary file?

Re: It Can Happen to You

#373

Post by komodo009 » Thu, Mar 04, 2021, 6:35 PM UTC

ignoramous wrote: post #262

Waterluvian wrote: post #66

Earlier quoted context omitted.

I love the format of this blog post. Perfect length. Perfect detail. You don't waste words. A good number of images.

I like this blog post but I also happen to quite like long forms like these, too: https://mango.pdf.zone/finding-former-australian-prime-minis... / https://apenwarr.ca/log/20201227

The mango blog post was incredibly entertaining. Thanks for sharing.

Re: It Can Happen to You

#374

Post by SilasX » Thu, Mar 04, 2021, 6:36 PM UTC

thaumasiotes wrote: post #363

SilasX wrote: post #351

Earlier quoted context omitted.

>The same approach applies to key length requirements for hash tables with arbitrarily large backing stores. They do not grow as slowly as the CLRS log* function, but they grow so slowly that there are easily identifiable sharp limits on how large they can be -- an easy example is that a hash table cannot use more memory than the hardware offers no matter how the software is written. A backing store with 1TB of addre…

> No, I was using table size exactly as you, to mean the number of elements stored. Is there a reason my comments only made sense under a different definition? It not, be charitable. (And avoid using obscure terms.) I interpreted your comment to refer to the size of the backing store, because that is fundamentally what a hash key needs to be able to address. I didn't mean to say that, if you were using it that way, y…

>I interpreted your comment to refer to the size of the backing store, because that is fundamentally what a hash key needs to be able to address.

Under the assumption (upthread) of constant resizing as element are added, the distinction is irrelevant. The more elements you have in the table, the more elements you need to address, and the more possible outputs your hash function needs to have.

And the needed size of the backing store scales with the number elements you want to store anyway.

>I didn't mean to say that, if you were using it that way, you were doing anything wrong, only that there appeared to be a mismatch.

Why bring up something like that if it doesn't translate into something relevant to the discussion e.g. to show my point to be in error?

Re: It Can Happen to You

#375

Post by froh » Thu, Mar 04, 2021, 6:38 PM UTC

moonchild wrote: post #56

(Originally on lobsters[0].) I maintain my original position that sscanf calculating the entire length of its input is absolutely ridiculous. Are *scanf difficult to use safely, not very robust, and somewhat baroque? Yes. Should sscanf("%f") be a correct (not performance-killing) way of reading floats? Also yes. (Though aside: the OP seems to be reading data from files, so they could have just used fscanf, which has…

most of these provide fmemopen(3), which makes the temporary FILE object explicit and persistent for the whole parse, and needs just one strlen call.

https://news.ycombinator.com/item?id=26343149

Re: It Can Happen to You

#376

Post by froh » Thu, Mar 04, 2021, 6:39 PM UTC

tyingq wrote: post #57

This HN comment and its follow ups are an interesting read. There are some stdlib implementations that don't have this issue. https://news.ycombinator.com/item?id=26298300

also a trivial workaround is to wrap the buffer with a FILE via fmemopen(3), which makes the temporary FILE object explicit and persistent for the whole parse, and needs just one strlen call.

https://news.ycombinator.com/item?id=26343149

Re: It Can Happen to You

#377

Post by kaba0 » Thu, Mar 04, 2021, 6:41 PM UTC

oblio wrote: post #294

kaba0 wrote: post #219

Earlier quoted context omitted.

The thing is, they are even worse for performance than string implementations that store the length.. that extra few bits of memory is much cheaper than checking the size of a string everywhere. For example, copying a string with known length. Also, c++’s strings even do some clever hacking where they store the text itself for shorter strings in the pointer, barring a pointer lookup. And this is possible only because…

They were designed when an extra byte or so per string cost you a lot of money. Nowadays, when 99% of the systems anyone will program start at 1MB RAM and 90% probably start at 512MB, they're a liability for almost no benefit.

You’ve got an extra byte either way, the \0 at the end. Which in many cases will make you copy a string because you can’t just “point” into a string literal and say take n chars from there. Of course I am not that old so I don’t have enough expertise — but seeming that every other language even at the time decided against it is pretty telling.

Re: It Can Happen to You

#378

Post by froh » Thu, Mar 04, 2021, 6:42 PM UTC

edflsafoiewq wrote: post #16

astrea wrote: post #13

It has been a hot minute since I've touched C, so I'm failing to grok the issue here. Sscanf is reading the data variable for a float-formatted string into a float variable. How is that also getting the size? What is different about strtof? It looks from the docs that it does something similar, just without using the formatting string.

> sscanf() converts the string you pass in to an _IO_FILE* to make the string look like a "file". This is so the same internal _IO_vfscanf() can be used for both a string and a FILE*. > However, as part of that conversion, done in a _IO_str_init_static_internal() function, it calls __rawmemchr (ptr, '\0'); essentially a strlen() call, on your input string. This conversion is done on every call to sscanf(), and since…

and the proper remedy is to use fmemopen(3), which makes the temporary FILE object explicit and persistent for the whole parse, and needs just one strlen call.

https://news.ycombinator.com/item?id=26343149

Re: It Can Happen to You

#379

Post by kaba0 » Thu, Mar 04, 2021, 6:45 PM UTC

akkartik wrote: post #221

kaba0 wrote: post #214

Earlier quoted context omitted.

Hi, level 1 and 2 looks really cool, but I may not understand the point of only having these languages “running” on a computer? Both 2 and 3 are interpreted and the interpreter is the area you want to minimize? What about a program written in 3 that can compile to either 1 or 2? Why would it hurt anything to have a different language somehow made possible to run here?

I'm not sure I follow your question, but the idea is that the computer the end user receives has a single opinionated language for programming it. Kinda like Basic for microcomputers. The end user is of course welcome to build their own languages. That is encouraged! But multiple languages make the computer more difficult for others to comprehend. My goal is to convince you that, all things being equal, a computer wi…

My question is regarding why would a single language codebase be easier to comprehend/have fewer bugs, security holes? In terms of a single program it makes sense, but I seldom read the source code of a library for example I depend on - if it has a good public API, it could be written in anything for all I care.

Not trying to dismiss the idea at all, just I don’t yet see “the light”, so to say.

Re: It Can Happen to You

#380

Post by froh » Thu, Mar 04, 2021, 6:48 PM UTC

simias wrote: post #320

I think the really embarrassing part for Rockstar is that they didn't bother to investigate what took 5+ minutes to load in their star product, a simple profiling would've made the issue obvious. So either they knew and they didn't care, or they didn't know and they didn't care. That being said both for GTA and for TFA the issue is a very similar sscanf call: sscanf(data, "%f", &f); I already posted a similar comment…

scanf and printf have complementary format specifiers, which can make maintaining serialization and parsing of regular data a breeze...

the proper remedy is to simply wrap the string to parse with fmemopen(3), which makes the temporary FILE object explicit and persistent for the whole parse, and needs just one strlen call.

https://news.ycombinator.com/item?id=26343149