It Can Happen to You • Hacker News Board

Re: It Can Happen to You

#401

Post by ajkjk » Fri, Mar 05, 2021, 2:06 AM UTC

beaconstudios wrote: post #250

ajkjk wrote: post #241

Earlier quoted context omitted.

But if you know the performance of an algorithm up front, you don't have to spend any time optimizing it in the first place. You just know what to do, because you know the performance. For instance: suppose you are building a CRUD app on a SQL database. Do you (a) add indexes for important queries as you go? or (b) ignore indexes and later profile and see what queries are slow. No, of course you just make the indexes…

Of course you index hot columns up front in that case, but I think where we disagree is that you want to generalise "optimise up front" into a rule, do or don't; I consider whether it's applicable in the circumstance. C programs tend to use a lot of system calls, and are also usually easily rapidly testable with large data. So rather than profile every individual std function I call, I'll just profile the very resour…

> I disagree with unilateralism

I mean, that's my point too. There's a camp of people who will say "don't prematurely optimize! profile and tune the hotspots later" as a blanket rule and I think that's dumb. And I thought you were espousing that.

Re: It Can Happen to You

#402

Post by desertrider12 » Fri, Mar 05, 2021, 2:36 AM UTC

iamevn wrote: post #180

desertrider12 wrote: post #140

Earlier quoted context omitted.

This isn't really related to your question, but I don't think tail calls could help for Fibonacci since f(n) branches to two calls, f(n-1) and f(n-2). And each of those branches into 2. So it can't be done in a finite stack area with naive recursion. The compiler would either have to memoize, or be extremely clever and start at the base case (0, 1) and then transform the code to use the 2x2 matrix exponentiation. I w…

Wouldn't this be the recursion version with tail calls? (define/contract (fib n) (-> nonnegative-integer? nonnegative-integer?) (let fib-recur ([i 1] [curr 1] [prev 0]) (cond [(= i n) curr] [else (fib-recur (+ i 1) (+ curr prev) curr)])))

That's pretty much what I meant by the matrix exponentiation method - applying the map (a, b) -> (a+b, a) repeatedly. Your function definitely uses tail calls, but I was just trying to say more than tail call optimization is needed to transform the trivial recursive version

fibb 0 = 0

fibb 1 = 1

fibb n = + (fibb (- n 1)) (fibb (- n 2))

into that, or any O(n) version.

Re: It Can Happen to You

#403

Post by beaconstudios » Fri, Mar 05, 2021, 3:03 AM UTC

ajkjk wrote: post #401

beaconstudios wrote: post #250

Earlier quoted context omitted.

Of course you index hot columns up front in that case, but I think where we disagree is that you want to generalise "optimise up front" into a rule, do or don't; I consider whether it's applicable in the circumstance. C programs tend to use a lot of system calls, and are also usually easily rapidly testable with large data. So rather than profile every individual std function I call, I'll just profile the very resour…

> I disagree with unilateralism I mean, that's my point too. There's a camp of people who will say "don't prematurely optimize! profile and tune the hotspots later" as a blanket rule and I think that's dumb. And I thought you were espousing that.

No no that's not my angle at all, but I'm glad we're in agreement. Clear communication is hard!

Re: It Can Happen to You

#404

Post by lanstin » Fri, Mar 05, 2021, 6:23 AM UTC

perl4ever wrote: post #78

tux3 wrote: post #15

The moral of the story, as far as I'm concerned: do NOT parse strings in C! Use a library, prefferably in a higher-level language. C string handling is a mess of viciously surprising APIs, juggling those particular footguns is almost certainly not your least bad option.

I used to work for people that processed emails and loaded them into databases with perl scripts. One day someone asked me if I could help, because the script they were running on a batch of emails was inexplicably freezing or running out of memory, I forget the exact details. There were maybe a few thousand or tens of thousands of emails, and so, I came to look at the issue with my usual attitude which is that if it…

There is a whole lot of low hanging fruit in the world. When I am new at a job if I don’t find several order or two of magnitude improvements I am impressed.

Re: It Can Happen to You

#405

Post by didibus » Fri, Mar 05, 2021, 6:49 AM UTC

arcturus17 wrote: post #383

didibus wrote: post #369

Earlier quoted context omitted.

> In computer science, a loop is a programming structure that repeats a sequence of instructions until a specific condition is met. That's the general definition at least I've always been most aware of. I don't want to claim it is the most common one, cause I don't really have numbers and who is the authority on comp-sci definitons? But I do feel it is at least a somewhat common academic definition for looping. That…

There's no denying that from that definition they are the same. It's just after you've debugged enough loops and recursions you can't help but think they are quite different!

Well, I don't mean they are the same, they're just different kinds of looping constructs.

Like what you call a loop isn't a loop, its actually a for-loop, or its a while-loop, or a for-each loop, or its an iterator loop, and similarly recursion is just a recursive loop.

At least that's the common taxonomy I know off. So all these are loops, and the ones that involve mutation for the condition to kick in are further grouped as imperative loops.

Re: It Can Happen to You

#406

Post by kaba0 » Fri, Mar 05, 2021, 7:24 AM UTC

akkartik wrote: post #394

kaba0 wrote: post #393

Earlier quoted context omitted.

Thanks for the answer! I totally agree on the not enough people read source code part — unfortunately I believe it is not only a language “barrier” thing. I mean, even in a language I know by heart, I probably could not make sense of some complex part of the linux kernel, because I lack both the underlying technical knowledge on some hardware interface, or the context of the code. And especially this latter can not b…

Yeah, level 3 will be a HLL. It just doesn't matter too much which one it is, or that it "rules them all". A single reasonably high-level language X is in practice superior to a basket of high-level languages, even if some of the languages in the basket are individually higher-level than X. You're absolutely right that languages are only part of the problem. Beyond the language choice, Mu provides guardrails to help…

Thank you for the discussion and good luck with your project!

Re: It Can Happen to You

#407

Post by raihansaputra » Fri, Mar 05, 2021, 7:38 AM UTC

dcolkitt wrote: post #371

specialist wrote: post #365

Earlier quoted context omitted.

You kid. But truer things are said in jest. > ...Tomorrow, someone’s going to reduce the boot times of macOS by 90% by the same principle. My 2019 MacBook often pauses when I connect the charging cable. Sometimes it just seizes, requiring a hard bounce. Clearly there's a contended lock buried deep. Something non-obvious. I'm certain everything these days has dozens of hidden quadratics and contended locks. Which is o…

> My 2019 MacBook often pauses when I connect the charging cable. Sometimes it just seizes, requiring a hard bounce. Yesterday my MBP kernel panicked because my keyboard was low on battery and the bluetooth connection kept dropping. There's something weird with MacOS where peripherals seem to really not be well isolated from the core OS runtime.

Oh peripherals on newer Macs are somehow very hit or miss. I have a very difficult time with external monitors, especially from sleep. My MBP 16" would just loop between initializing and failing to initialize, until I unplug, wait, and re-plug again. Or I have to press the `Extend` option instead of the `Mirror` option that I use. The older 2015 MBP would just connect fine.

Re: It Can Happen to You

#408

Post by Chyzwar » Fri, Mar 05, 2021, 8:19 AM UTC

IggleSniggle wrote: post #349

DarkWiiPlayer wrote: post #229

Earlier quoted context omitted.

Good documentation and inspecting the compiled bytecode are both good ways of finding out about performance characteristics of certain features. The problem starts when people rely on assumptions ("sscanf should be fast because it's widely used") or performance folklore ("localizing every function you'll ever use makes your Lua code faster"), because those tend to either be completely wrong or lack very important con…

I live in js land, and the barrier between “folklore” and “documentation” is extremely thin. Especially since V8 may introduce changes at any time that affect performance characteristics of js. I’d respond with “well if performance matters it shouldn’t be in js” except for all the shite being written in js these days, with js being the hammer that makes everything else look like a nail.

V8 documents these changes very well[1].

You can write very fast JS code. When carefully written it can have Java like performance[2]. It is just very hard in practice where most ecosystem is optimized for developer productivity.

When performance matter, write your own code and carefully benchmark everything. You can see this working for Typescript and VSCode[3]

[1] https://v8.dev/blog [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [3] https://github.com/microsoft/TypeScript/pull/43035#issuecomm...

Re: It Can Happen to You

#409

Post by throwaway81523 » Fri, Mar 05, 2021, 8:21 AM UTC

I think the real moral is that if you're doing something where performance matters even slightly, profile it and see if the time is being spent where you expect it to be spent. If not, investigate and fix as required.

Re: It Can Happen to You

#410

Post by TeMPOraL » Fri, Mar 05, 2021, 8:22 AM UTC

marton78 wrote: post #372

TeMPOraL wrote: post #199

Earlier quoted context omitted.

You're joking, but now I'm thinking about the XML we parse at work and the library we're using to do it. We parse a lot of it, but I've always had this vague feeling that it takes a bit too long (given the codebase is C++). The XML library we use is rather well-known, so if someone found a bug like this there, I'd suspect a general improvement of performance across the board in the entire industry. Efficient Market H…

What about parsing that XML upfront, serialising to some binary format (e.g. CBOR, maybe with nlohmann's JSON library, or Cap'n Proto) and shipping the binary file?

Would be cool if we could that, but as things stand, enough various people want to occasionally look at these files, in environments where they can't just install specialized tooling and are using notepad.exe (or Notepad++ if already available), that we keep it text.

I like binary formats, but we can't afford the increased complexity around supporting a custom binary format, so I'm not pushing for changes here.

I did investigate replacing our pile of XML files with an SQLite database, which would give us fast and efficient format, and allow to use existing SQLite database viewers, or hit the file with trivial scripts, so we'd have no complexity supporting a dedicated tool. However, the data model we use would need such a large overhaul (and related retraining) that we tabled this proposal for now.