Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Recording and visualising the 20k system calls it takes to "import seaborn"

blog.mattstuchlik.com

Post reply on HN
Print view View original

21–30 of 56 posts

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#21

Post by josephg »

dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

Yeah. I recently worked on a small web project being developed at a university. The project is written in flask, and it presents a reasonably simple UI on top of some data living in a mysql database.

When I started on the project, page loads often took 10 seconds or more. The web application is used by about 20 people and that was enough to bring their single beefy server to its knees. Someone in NY tried scraping the site the other week and the site became completely unresponsive. They resorted to banning the IP to keep the website up. The reasons it was slow were all the usual culprits - a misused ORM being the main one.

It’s a nice language, but I really felt like I’d been transported back in time a few decades working in it. It feels like I’m using a computer from the 90s where performance choices matter again because the language is so slow. And where dependency management is a circus of half working tools and half hearted attempts at versioning. Packages conflict with one another. Some “pinned” package versions have apparently rusted and won’t actually install on my computer. And the system to install packages locally was obviously bolted on, badly, long after the horse had left the gate.

It reminds me of working in C in the early 2000s. I never thought I’d say this but it makes server side JavaScript with npm look positively modern and fast by comparison.

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#22

Post by jiggawatts »

turtlebits wrote: post #20
albertgoeswoof wrote: post #9

Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.

Implementing traces should be done by devs, not the infrastructure team. Devops should implement/support the platform that supports traces.

In Azure, use Application Insights for this. It's easy to set up and shows distributed performance traces across multiple services in a GUI.

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#23

Post by floating-io »

pdonis wrote: post #17
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

> I'm going to go back to learning more C and Forth Why would you expect that to decrease the number of syscalls you need? The syscalls are there because the program needs the OS to do things. That need is driven by the application domain, not by the programming language you use.

Unless the programming language you use happens to perform 20,000 system calls before it ever even runs a line of your actual code...

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#24

Post by teddyh »

QuercusMax wrote: post #16
jiggawatts wrote: post #13

Earlier quoted context omitted.

I keep telling people that a hundred buses will let you take a hundred times more people, but nobody will get to their destination a hundred times faster. Inevitably this comment is followed by quiet blinking as they digest this and then this question: “Are you saying we need to scale up one hundred times bigger?” Sigh…

Really makes one wonder what kind of thought process these people go through, and what kind of education they actually had.

On two occasions I have been asked, — “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?” In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

— Charles Babbage, Passages from the Life of a Philosopher (1864), chapter 5, Difference Engine No. 1

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#25

Post by pixelesque »

pdonis wrote: post #17
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

> I'm going to go back to learning more C and Forth Why would you expect that to decrease the number of syscalls you need? The syscalls are there because the program needs the OS to do things. That need is driven by the application domain, not by the programming language you use.

Python's module importing / $PYTHONPATH lookup/traversal is incredibly inefficient, especially with cold FS caches...

I've worked at places where we've significantly patched the logic (in a way which breaks compatibility in some cases, so couldn't be up-streamed) which makes Python startup / module loading with hundreds of paths in $PYTHONPATH orders of magnitude faster...

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#26

Post by amluto »

josephg wrote: post #21
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

Yeah. I recently worked on a small web project being developed at a university. The project is written in flask, and it presents a reasonably simple UI on top of some data living in a mysql database. When I started on the project, page loads often took 10 seconds or more. The web application is used by about 20 people and that was enough to bring their single beefy server to its knees. Someone in NY tried scraping th…

I once worked on a small web project, at a university, in Python, using WSGI IIRC. It loaded a lot faster than any of the big expensive apps the university had written.

Well, there was one exception. The little import statement to import the Oracle database client took maybe 15 seconds. MySQL for the win :)

(I would not recommend MySQL for new applications today, although I might recommend it over Oracle…)

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#27

Post by [deleted] »

josephg wrote: post #21
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

Yeah. I recently worked on a small web project being developed at a university. The project is written in flask, and it presents a reasonably simple UI on top of some data living in a mysql database. When I started on the project, page loads often took 10 seconds or more. The web application is used by about 20 people and that was enough to bring their single beefy server to its knees. Someone in NY tried scraping th…

[deleted]

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#28

Post by throwaway828 »

josephg wrote: post #21
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

Yeah. I recently worked on a small web project being developed at a university. The project is written in flask, and it presents a reasonably simple UI on top of some data living in a mysql database. When I started on the project, page loads often took 10 seconds or more. The web application is used by about 20 people and that was enough to bring their single beefy server to its knees. Someone in NY tried scraping th…

That sounds like some quick kills to be easily made.

I use a dev machine that's quite archaic compared to a modern server, a 2nd gen i5 ThinkPad to be precise, that struggles to top 20ms for a request including loading a user and data object, joined tables and all, via ORM from Postgres running locally with a few hundred thousand records in said tables, before touching anything like explicitly adding caching.

Check your indexes, joins, general DB design and in-app looping. Flask's not your problem. You'll have equal or worse woes (if lower level with less hand holding) with anything else.

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#29

Post by nonethewiser »

josephg wrote: post #21
dmwilcox wrote: post #2

I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view. I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)

Yeah. I recently worked on a small web project being developed at a university. The project is written in flask, and it presents a reasonably simple UI on top of some data living in a mysql database. When I started on the project, page loads often took 10 seconds or more. The web application is used by about 20 people and that was enough to bring their single beefy server to its knees. Someone in NY tried scraping th…

Im curious what the orm misuse was because Python can obviously handle a lot higher loads than that. Perhaps the orm is to blame for offering some footgun. Or maybe the developer did something impossibly idiotic.

Re: Recording and visualising the 20k system calls it takes to "import seaborn"

#30

Post by nsm »

Related to this, if you set the env var `PYTHONPROFILEIMPORTTIME=1` or run python with `-X importtime`, it will print out the cumulative and self times to import various modules.

There is then this neat tool to visualize the data. https://kmichel.github.io/python-importtime-graph/

Highly recommend to find the worst imports affecting your program startup time.

In general, the python community values tend towards functionality over performance. For example, large modules (looking at networkx here) will often import a bunch of there submodules in their __init__.py, which means all modules now end up loaded even if you didn't need them.

I've never tried https://pyoxidizer.readthedocs.io/en/stable/oxidized_importe..., but it compiles all the imports into one, memory mapped file, that _may_ speed up the importing.

Having everything compiled to bytecode also helps a bunch.

Post reply on HN