11–20 of 56 posts
I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)
Earlier quoted context omitted.
It doesn’t take 20k syscalls to print a plot, the 20k syscalls is for the import call. I would hope that drawing plots takes a lot less. To engage with your point: loading a dynamic library in a regular language takes significantly less than 20k syscalls. Probably 20-40 for C on Linux. Python is uniquely inefficient. On most plots comparing resource use by different languages, in order to even show python together wi…
Modularity and customization come at a cost. Python is the systemd of computer languages. But it is not trying to sell itself under the KISS banner.
It just worries me when I sometimes see those same Jupyter notebooks running in production, crunching 100s of terabytes of data. Maybe I’m wrong, but I didn’t get the impression everyone realizes exactly how wasteful that is. I guess AWS credits are easy to come by.
One thing Google did well back in the day, was making resource costs report in SWE/hours, the idea being that you see if you should go and rewrite something. If it cost 100 SWE/h to run, and it only took you a day to cut that in half, you should do it.
Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.
Inevitably this comment is followed by quiet blinking as they digest this and then this question: “Are you saying we need to scale up one hundred times bigger?”
Sigh…
Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.
Also, with DevOps pushing out traditional administrators, companies are often spending way more on infra than needed.
Earlier quoted context omitted.
Modularity and customization come at a cost. Python is the systemd of computer languages. But it is not trying to sell itself under the KISS banner.
Oh absolutely, for some tasks Python is amazing. I use Jupyter notebooks a lot, for example, and the flexibility is an incredible feature. It just worries me when I sometimes see those same Jupyter notebooks running in production, crunching 100s of terabytes of data. Maybe I’m wrong, but I didn’t get the impression everyone realizes exactly how wasteful that is. I guess AWS credits are easy to come by. One thing Goog…
Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.
I keep telling people that a hundred buses will let you take a hundred times more people, but nobody will get to their destination a hundred times faster. Inevitably this comment is followed by quiet blinking as they digest this and then this question: “Are you saying we need to scale up one hundred times bigger?” Sigh…
I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)
Why would you expect that to decrease the number of syscalls you need? The syscalls are there because the program needs the OS to do things. That need is driven by the application domain, not by the programming language you use.
Earlier quoted context omitted.
So, instead of printing a pretty plot in 2 lines of code, you will be... making these 20k syscalls yourself? You have such unusual hobby, my friend!
It doesn’t take 20k syscalls to print a plot, the 20k syscalls is for the import call. I would hope that drawing plots takes a lot less. To engage with your point: loading a dynamic library in a regular language takes significantly less than 20k syscalls. Probably 20-40 for C on Linux. Python is uniquely inefficient. On most plots comparing resource use by different languages, in order to even show python together wi…
Yes, because you're importing a library that does a lot more than just print a plot. A purpose-built Python program that just printed the plot, nothing else, would need a lot less than 20k syscalls too.
Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.