Most algorithm analyses don't incorporate memory hierarchies. I'm not sure what the point of this post was. If you are really concerned about it, compute the empirical roofline for your machine. Also, if the point is to point out memory hierarchies, it's not 'quadratic performance.' The algorithm doesn't behave differently once it spills over. The costs just get bigger.
Python sets and dictionaries can have quadratic-time performance
41–50 of 67 posts
Re: Python sets and dictionaries can have quadratic-time performance
#42Real big brain moment in the comments >Can we say that since every real-life data structure size is bounded by some constant, it is O(1)? If not, why? >>You may, yes. Very powerful thinking coming from someone who is "a software performance expert. He ranks among the top 2% of scientists globally (Stanford/Elsevier 2025) and is one of GitHub's top 1000 most followed developers"
It's true you can. Just as you can say that you can enumerate all memory states in a real computer so it can be modelled as a finite state machine. These asymptotic models aren't real (our world as we experience it is finite and bounded) and exist as models from which to gain insights which we can transfer back. I love this argument BTW a classic that usually comes up in these discussions.
Re: Python sets and dictionaries can have quadratic-time performance
#43Generally for performance, the rule is you _measure_ it, you don't rely on big-O. So if you're really concerned, please just use a profiler. Also hashmaps (python Dicts) have amortized O(1) and not O(1) big-O.
Re: Python sets and dictionaries can have quadratic-time performance
#44Re: Python sets and dictionaries can have quadratic-time performance
#45Earlier quoted context omitted.
It's true you can. Just as you can say that you can enumerate all memory states in a real computer so it can be modelled as a finite state machine. These asymptotic models aren't real (our world as we experience it is finite and bounded) and exist as models from which to gain insights which we can transfer back. I love this argument BTW a classic that usually comes up in these discussions.
Asymptotic models are not a statement of how long it will take; they are classes of functions modeling the relationship between some performance outcome (run time most often, but sometimes memory or other things) and data coming in. These mappings from input to performance characteristics are not conditioned on physical limitations but are rather abstract tools for measuring that relationship.
Re: Python sets and dictionaries can have quadratic-time performance
#46Which statement in this article applies only to Python?
The part where he chooses his inputs to hit worst case behavior in Python's hash function.
M = (1
which are effectively random from the hash function's point of view, especially with a randomized seed (the default on current versions).Re: Python sets and dictionaries can have quadratic-time performance
#47Earlier quoted context omitted.
Asymptotic models are not a statement of how long it will take; they are classes of functions modeling the relationship between some performance outcome (run time most often, but sometimes memory or other things) and data coming in. These mappings from input to performance characteristics are not conditioned on physical limitations but are rather abstract tools for measuring that relationship.
If you have an instance (which is what I assume is implied by "real-life data structure") bounded by a particular size, then you could say that the runtime complexity is constant with respect to that bound. Whether that is a useful statement to make is another discussion entirely. On its own probably not. It can potentially be useful for analyzing the runtime complexity of operations in more complex algorithms that u…
There's another level of imprecision here in common usage, which is that big-O is strictly an upper bound, meaning that Merge Sort is O(N!). What they really mean is big-Theta, a "tight" bound, in which Merge Sort would be θ(n log n).
But for simplicity's sake, let's use big-O to mean "tightly bound" like people do in casual discussion, and further let's say we're talking about runtime as the size of the function based on its domain:
There is no such thing as big-O performance bounded by a "real-life data structure". The entire point of big-O is that it is asymptotic analysis. You could define a number H, where H is the time to the heat death of the universe, and Python's sets and dictionaries would never be O(H). Because you could for your set/dict runtimes of f(n) still find a constant C and k such that f(n) > C*H for n > k. One example would be setting k to H*C + 1, and that works for either f(n) ∈ O(1) or f(n) ∈ O(n^2).
You can analyze algorithms with respect to real physical limitations, but at that point you are not talking about their "big-O" performance. So the author, despite being "top 2% of scientists" in his field, which is "software performance", seems to still be lagging behind your average freshman compsci student who crammed for their complexity analysis exam.
Re: Python sets and dictionaries can have quadratic-time performance
#48Reminds me of the article about random memory access being O(√N): https://www.ilikebigbits.com/2014_04_21_myth_of_ram_1.html
Re: Python sets and dictionaries can have quadratic-time performance
#49Raymond Hettinger has a great talk about how much python's dict has improved over the years. So this is super interesting and will probably just make the builtin dict better eventually. The lesson of the talk is that if you are idiomatic then you will benefit as the language improves. https://www.youtube.com/watch?v=npw4s1QTmPg
Re: Python sets and dictionaries can have quadratic-time performance
#50Uh, what is going on with this benchmark? Why is M so big? Why does it cross the maxint boundary? Why is constructing the list comprehension part of the benchmark? Why are we summing the set? Why are we only measuring 5 values for n?
`values = [str(i * M) for i in range(1, n + 1)]`
or
`values = [i * M % 1_000_000_000_000_000 for i in range(1, n + 1)]`