Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weird Python Integers

kate.io

Post reply on HN
Print view View original

21–30 of 150 posts

Re: Weird Python Integers

#21

Post by boramalper »

> We can use the Python built-in function id which returns a value you can think of as a memory address to investigate.

> [...]

> It looks like there is a table of tiny integers and each integer is takes up 32 bytes.

It is the memory address but it's a "CPython implementation detail: This [return value of the id() function] is the address of the object in memory."[1]

Though you cannot use this to determine the size of an object, or rather you "shouldn't" because that assumes a very specific implementation detail, which isn't there.

If you'd like to get the size of an object, use sys.getsizeof().[2] Also keep in mind that containers in Python does not contain the objects themselves but references to them so the returned size is the size of the object itself only, non-recursively. Read "Is Python call-by-value or call-by-reference? Neither."[3] for some more details.

[1]: https://docs.python.org/3.6/library/functions.html#id

[2]: http://docs.python.org/3.6/library/sys.html#sys.getsizeof

[3]: https://jeffknupp.com/blog/2012/11/13/is-python-callbyvalue-...

Re: Weird Python Integers

#25

Post by mbell »

Ruby does something similar, but all Fixnum (native sized) values are 'fixed objects':

    a = 2**62 - 1
    b = 2**62 - 1

    a.object_id == b.object_id # true

    a = 2**62
    b = 2**62

    a.object_id == b.object_id # false
Ruby does automatic promotion from Fixnum (native size) to Bignum (arbitrarily large) and uses one bit of the native size as a flag to identify this which is why 2^62 - 1 is the max instead of 2^63 - 1. Though I think this is only true of MRI and other implementations handle it without the flag bit.

Perhaps one difference from Python is that in MRI Ruby Fixnum doesn't really even allocate an 'object', the object_id is the value in disguise. In fact all 'real' objects have even object_ids and all odd object_ids are integers:

    a = 123456789
    (a.object_id - 1) / 2 # 123456789

Re: Weird Python Integers

#26

Post by jwilk »

masklinn wrote: post #7
woodrowbarlow wrote: post #5

the python documentation [1] says the following: > The current implementation keeps an array of integer objects for all integers between -5 and 256, when you create an int in that range you actually just get back a reference to the existing object. So it should be possible to change the value of 1. I suspect the behaviour of Python in this case is undefined. :-) does anyone have any idea how they chose that range? it…

You could probably find out from the code's log. I'd guess the small positive integers are common due to e.g. iteration or len() of small collections and the like, and the very small negatives are due to things like error values.

I did some git archaeology. Here are relevant commits and bug reports:

* Initial commit with -1...99 cached:

https://github.com/python/cpython/commit/842d2ccdcd540399501...

* Cache more negative numbers, -5...99:

https://github.com/python/cpython/commit/c91ed400e053dc9f11d...

https://bugs.python.org/issue561244

* Cache more positive numbers, -5...256:

https://github.com/python/cpython/commit/418a1ef0895e826c65d...

https://bugs.python.org/issue1436243

Re: Weird Python Integers

#27

Post by dvfjsdhgfv »

wyldfire wrote: post #9

> That is suprising! It turns out that all “small integers” with the same value point to the same memory. We can use the Python built-in function id which returns a value you can think of as a memory address to investigate. Unfortunately this blog post seems to miss a great opportunity to show you how you should compare integers for equality -- using the equality operator `==` and not the identity comparison `is`. ED…

Frankly, what's the point? This is common knowledge.

Re: Weird Python Integers

#28

Post by int_19h »

boramalper wrote: post #21

> We can use the Python built-in function id which returns a value you can think of as a memory address to investigate. > [...] > It looks like there is a table of tiny integers and each integer is takes up 32 bytes. It is the memory address but it's a "CPython implementation detail: This [return value of the id() function] is the address of the object in memory."[1] Though you cannot use this to determine the size o…

Python memory model is amazingly simple and consistent: everything is passed and stored by value, but all values are references to objects.

Re: Weird Python Integers

#29

Post by sevensor »

santiagobasulto wrote: post #14

Nice article. I wrote a similar piece some time ago related to booleans, in case anybody is interested: https://blog.rmotr.com/those-tricky-python-booleans-2100d5df... And to avoid issues with is/==, we recommend our students to always use == (except for `is None`). Also related piece: https://blog.rmotr.com/avoiding-being-bitten-by-python-161b0...

That's unnecesarily restrictive -- I've gotten great use out of "is". You wouldn't want to use it on an integer, but I've been using namedtuples a lot lately, and "LastKnownConfiguration is CurrentConfiguration" works great to to check whether anything has changed without checking all of the fields for equality.

Re: Weird Python Integers

#30

Post by masklinn »

llccbb wrote: post #10
masklinn wrote: post #7

Earlier quoted context omitted.

You could probably find out from the code's log. I'd guess the small positive integers are common due to e.g. iteration or len() of small collections and the like, and the very small negatives are due to things like error values.

The small negatives might also be for indexing from the end of lists. `my_list[-1]` is super common.

Good point.
Post reply on HN