Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python numbers every programmer should know

mkennedy.codes

Post reply on HN
Print view View original

141–150 of 191 posts

Re: Python numbers every programmer should know

#141

Post by notepad0x90 »

thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

For some of these, there are alternative modules you can use, so it is important to know this. But if it really matters, I would think you'd know this already?

For me, it will help with selecting what language is best for a task. I think it won't change my view that python is an excellent language to prototype in though.

Re: Python numbers every programmer should know

#142

Post by sjducb »

lifeisstillgood wrote: post #86
sjducb wrote: post #85

It’s missing the time taken to instantiate a class. I remember refactoring some code to improve readability, then observing something that was previously a few microseconds take tens of seconds. The original code created a large list of lists. Each child list had 4 fields each field was a different thing, some were ints and one was a string. I created a new class with the names of each field and helper methods to pro…

I went to the doctor and I said “It hurts when I do this” The doctor said, “don’t do that”. Edit: so yeah a rather snarky reply. Sorry. But it’s worth asking why we want to use classes and objects everywhere. Alan Kay is well known for saying object orientated is about message passing (mostly by Erlang people). A list of lists (where each list is four different types repeated) seems a fine data structure, which can b…

The main reason why is to keep a handle on complexity.

When you’re in a project with a few million lines of code and 10 years of history it can get confusing.

Your data will have been handled by many different functions before it gets to you. If you do this with raw lists then the code gets very confusing. In one data structure customer name might be [4] and another structure might have it in [9]. Worse someone adds a new field in [5] then when two lists get concatenated name moves to [10] in downstream code which consumes the concatenated lists.

Re: Python numbers every programmer should know

#143

Post by saagarjha »

thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

I do performance optimization for a system written in Python. Most of these numbers are useless to me, because they’re completely irrelevant until they become a problem, then I measure them myself. If you are writing your code trying to save on method calls, you’re not getting any benefit from using the language and probably should pick something else.

Re: Python numbers every programmer should know

#144

Post by anhner »

IshKebab wrote: post #116
nutjob2 wrote: post #52

Earlier quoted context omitted.

> A lot of important and large codebases were grown and maintained in Python How does this happen? Is it just inertia that cause people to write large systems in a essentially type free, interpreted scripting language?

Someone says "let's write a prototype in Python" and someone else says "are you sure we shouldn't use a a better language that is just as productive but isn't going to lock us into abysmal performance down the line?" but everyone else says "nah we don't need to worry about performance yet, and anyway it's just a prototype - we'll write a proper version when we need to"... 10 years later "ok it's too slow; our options…

If I made an app in python and in 10 years it grows so successful that it needs a $10m vertical scale or $5m rewrite, I wouldn't even complain.

Re: Python numbers every programmer should know

#146

Post by superlopuh »

superlopuh wrote: post #145

I'm surprised that the `isinstance()` comparison is with `type() == type` and not `type() is type`, which I would expect to be faster, since the `==` implementation tends to have an `isinstance` call anyway.

Also seems like the repo is now private, so I can't open an issue, or reproduce the numbers.

Re: Python numbers every programmer should know

#147

Post by its-summertime »

dekhn wrote: post #115
NekkoDroid wrote: post #89

Earlier quoted context omitted.

> What exactly do you need 1ms instead of 14ms startup time in a shell startup? I think as they said: when dynamically building a shell input prompt it starts to become very noticable if you have like 3 or more of these and you use the terminal a lot.

Ah, I only noticed the "shell startup" bit. Yes, after 2-3 I agree you'd start to notice if you were really fast. I suppose at that point I'd just have Gemini rewrite the prompt-building commands in Rust (it's quite good at that) or merge all the prompt-building commands into a single one (to amortize the startup cost).

https://starship.rs/ perhaps? I should probably start using it again honestly.

Re: Python numbers every programmer should know

#148

Post by kristianp »

andai wrote: post #120

The one I noticed the most was import openai and import numpy. They're both about a full second on my old laptop. I ended up writing my own simple LLM library just so I wouldn't have to import OpenAI anymore for my interactive scripts. (It's just some wrapper functions around the equivalent of a curl request, which is honestly basically everything I used the OpenAI library for anyway.)

I have noticed how long it takes to import numpy. It made rerunning a script noticably sluggish. Not sure what openai's excuse is, but I assume numpy's slowness is loading some native dlls?

Re: Python numbers every programmer should know

#149

Post by Lockal »

ewuhic wrote: post #31

This is AI slop.

Sad that your comment is downvoted. But yes, for those who need clarification:

1) Measurements are faulty. List of 1,000 ints can be 4x smaller. Most time measurements depend on circumstances that are not mentioned, therefore can't be reproduced.

2) Brainrot AI style. Hashmap is not "200x faster than list!", that's not how complexity works.

3) orjson/ujson are faulty, which is one of the reasons they don't replace stdlib implementation. Expect crashes, broken jsons, anything from them

4) What actually will be used in number-crunching applications - numpy or similar libraries - is not even mentioned.

Re: Python numbers every programmer should know

#150

Post by Liquid_Fire »

CmdrKrool wrote: post #100

I'm confused by this: String operations in Python are fast as well. f-strings are the fastest formatting style, while even the slowest style is still measured in just nano-seconds. Concatenation (+) 39.1 ns (25.6M ops/sec) f-string 64.9 ns (15.4M ops/sec) It says f-strings are fastest but the numbers show concatenation taking less time? I thought it might be a typo but the bars on the graph reflect this too?

Perhaps it's because in all but the simplest cases, you need 2 or more concatenations to achieve the same result as one single f-string?

  "literal1 " + str(expression) + " literal2"
vs

  f"literal1 {expression} literal2"
The only case that would be faster is something like: "foo" + str(expression)
Post reply on HN