Earlier quoted context omitted.
Is there any practical programming language that is memory safe in its "entirety"? Python, for example, certainly is not. It has unsafe escape hatches (via ffi, at the very least). Yet, everyone I know of says and thinks of Python as a memory safe language. I do as well. > which makes it easier for developers to compartmentalize code to achieve memory-safety The problem here is that this is incomplete. Many many many…
> It has unsafe escape hatches (via ffi, at the very least). Yep, ctypes is part of the stdlib and lets you corrupt the VM on the fly. Fun stuff like changing the value of cached integers and everything. But ctypes being a terrifying pain in the ass, people tread very carefully around it. Cffi’s a lot better though it requires an external package. At the end of the day I think I’d be more enclined to bind through pyo…
I'm not sure how much people treading carefully actually translates into safety in practice.
CPython in particular has ad-hoc refcounting semantics where references can either be borrowed or stolen and you have to carefully verify both the documentation and implementation of functions you call because it's the wild west and nothing can be trusted: https://docs.python.org/3.9/c-api/intro.html#reference-count...
This ad-hoc borrowed vs stolen references convention bleeds into cffi as well. If you annotate an FFI function as returning `py_object`, cffi assumes that the reference is stolen and thus won't increment the ref count. However, if that same function instead returns a `struct` containing a `py_object`, cffi assumes the reference is borrowed and will increment the ref count instead.
So a harmless looking refactoring that changes a directly returned `py_object` into a composite `struct` containing a `py_object` is now a memory leak.
Memory leaks aren't so bad (even Rust treats them as safe after the leakpocalypse [1] [2]). It's when you go the other way and treat what should have been a borrowed reference as stolen that real bad things happen.
Here's a quick demo that deallocates the `None` singleton:
Python 3.9.13 (main, May 17 2022, 14:19:07)
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getrefcount(None)
4584
>>> import ctypes
>>> ctypes.pythonapi.Py_DecRef.argtypes = [ctypes.py_object]
>>> for i in range(5000):
... ctypes.pythonapi.Py_DecRef(None)
...
0
0
0
0
0
[snip]
Fatal Python error: none_dealloc: deallocating None
Python runtime state: initialized
Current thread 0x00007f28b22b7740 (most recent call first):
File "", line 2 in
fish: Job 1, 'python3' terminated by signal SIGABRT (Abort)