If you're working on Django codebases, `select_related` and `prefetch_related` are going to be your friends. Maybe 70-80% of Django slowness I've encountered (back when I did Django) was because of something issuing hundreds of db calls unnecessarily.
Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
11–20 of 26 posts
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#12Gallery code that queries all photos and shows them to the user. It had a 13 sec delay with 10k photos and above.
I thought it was a Big O problem at first, because the code was hacky and used more Arrays that it needed. But it was because it was obtaining all images and then sorting them by time.
I sped it up to milliseconds by making the queries sorted by time.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#13Wrapping multiple SQL mutations into a single transaction.
Adding the right indexes in a relational database.
Converting from csv to parquet before querying large datasets on Apache Spark.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#14Moved data aggregation from Oracle to `awk`
The process was loading 40GB files to database and aggregation took more than 5 hours.
Wrote a simple awk one liner with associate array and process was completed in minutes
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#15Almost twenty years ago, found a debug logging function using O_SYNC. Took it out. Test ran so fast that a teammate said that they thought it didn't run.
Now, of course, there's zero reason to write every debug output with O_SYNC. Classical case of cargo-cult programming. I'd like to say I've never seen something like that again, but then I'd be lying.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#16My most recent example is some SQL commands that were being executed on a nightly basis by another team against our database. Their code was sometimes taking more than eight hours to run, and this could cause timing problems where it took so long that it caused further processes down the pipeline to fail.
I took their exact SQL commands and wrapped them in my own scripting, and the simplified single-threaded version was executing in about fifteen minutes.
I went back through and added some explicit parallelization combined with wait commands to ensure that everything in that stage was complete before going to the next stage. That improved version now executes in around 600 seconds.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#17Does somebody have any stories for Python codebases?
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#18The company I used to work with was a b2b portal. Their Chinese customers were having issue to submit form due to the firewall and was taking 22 seconds. What I simply did was sending close header from server which made client browser to shut the request and release. Behind the scene it was still taking same time but at least customer weren't waiting. It was kind of Ajax but using apache flags.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#19Started a new job working on a company's API team. The API had out of memory issues and had processes crashing all the time. The code was PHP. All API calls ended like this: echo json_encode($data) . "\n"; Changed just one character, the period to a comma so the string wasn't duplicated before being output. Problem solved. Felt like a hero.
You probably found a co-worker's underhanded speed-bump meant to be taken out before the next annual performance review.
Re: Ask HN: Largest speedup you ever achieved by only changing a few lines of code?
#20If you're working on Django codebases, `select_related` and `prefetch_related` are going to be your friends. Maybe 70-80% of Django slowness I've encountered (back when I did Django) was because of something issuing hundreds of db calls unnecessarily.
To be fair, the django ORM makes it way too easy to do tens of thousands of queries and much too hard to collapse those down into join statements. select_related and friends will only get you so far.