Null References: The Billion Dollar Mistake • Hacker News Board

Re: Null References: The Billion Dollar Mistake

#41

Post by zozbot234 » Sat, Jan 11, 2020, 2:47 PM UTC

amelius wrote: post #34

zozbot234 wrote: post #12

Earlier quoted context omitted.

> Rust seems to be the only one that mostly circumvents this problem. The Rust hype is getting ridiculous here. There are plenty of languages with non-nullable references as first-class, and optionals for the nullable case. (...And I say this as a Rust fan myself, for what it's worth.)

Imho, Rust is an awkward language because it positions itself as a systems language but it makes low-level stuff more difficult (there's even a book teaching how to implement doubly linked lists in Rust [1]), hence prone to mistakes. At the same time, people are using Rust to build non-systems programs, where other languages would be more appropriate (e.g. those with garbage collectors). I don't think it is a good id…

> there's even a book teaching how to implement doubly linked lists in Rust [1]

Doubly-linked lists are an awkward example because the "safety" of a doubly-linked list as a data structure involves fairly complex invariants that Rust can't even keep track of at this point, much less check independently. These things are exactly why the unsafe{} escape-hatch exists and is actively supported. But just looking at the amount of unsafe code in common Rust projects should suffice to figure out that this is not the common case, at all.

> At the same time, people are using Rust to build non-systems programs, where other languages would be more appropriate (e.g. those with garbage collectors).

Garbage collectors are good for one thing, and one thing only: keeping track of complex, spaghetti-like reference graphs where cycles, etc. can arise, perhaps even as a side effect of, say, implementing some concurrency-related pattern. Everything else is most likely better dealt with by a Rust-like system with optional support for reference counted data.

That's without even mentioning the other advantages that a Rust-like ownership system provides over a GC-only language. See e.g. https://llogiq.github.io/2020/01/10/rustvsgc.html this recent post for some nice examples.

Re: Null References: The Billion Dollar Mistake

#42

Post by zozbot234 » Sat, Jan 11, 2020, 2:54 PM UTC

DaiPlusPlus wrote: post #17

seventh-chord wrote: post #15

Earlier quoted context omitted.

in languages like c, rust or go, where you can put arbitrary data on the stack, it seems to me as if such issues are less common because you dont have to worry about initializing pointers and allocating memory unless you actually want to put something on the heap. Thus if you make everything a reference in your language its no wonder you run into issues like null-pointers more often

With stack allocation you then encounter problems with object lifetime. Rust solves this problem by binding references to scope, and Go solves this by invisibility changing an allocation to the heap (and uses ref-counting? I think?). I wish C had a feature that would let you allocate something on the stack and then return to the parent stack frame without popping the stack-pointer - that would be handy for self-conta…

Go uses fully-general GC, not reference counting. Obligate reference counting is used in other languages such as Swift, probably with worse throughput than obligate tracing GC.

Re: Null References: The Billion Dollar Mistake

#43

Post by JakobProgsch » Sat, Jan 11, 2020, 2:56 PM UTC

auxym wrote: post #20

dorfsmay wrote: post #14

Earlier quoted context omitted.

Everything in Python is a reference, and there's no null pointer issues.

I've certainly had some "None" errors in Python. I think the difference comes from dynamic vs static typing. In Python, you sort of get into the habit of "defensive" programming: checking inputs to your function, catching Nones, etc. In java, you tend to rely more on the type system. If it typechecks/compiles, there's a good chance it's OK. That is, until you get a null value that's not handled. That's the root issue…

The first line is why I'm not 100% convinced of the severity of this mistake compared to the alternatives. The problem fundamentally is the use of magic values/numbers to represent the concept of "no value". You don't need explicit language support to have that concept and the bugs it causes. I guess having that as an intrinsic concept in the language makes it more likely that people use it badly. On the other hand debuggers etc. also intrinsically understand this and segfaults due to null pointers are usually very easy to localize once you see them. On the other hand if a "bad programmer" introduced their own magic non-value in a supposedly safe language, debugging that becomes way more confusing.

Re: Null References: The Billion Dollar Mistake

#44

Post by agumonkey » Sat, Jan 11, 2020, 3:01 PM UTC

augusto2112 wrote: post #40

agumonkey wrote: post #39

Earlier quoted context omitted.

Null is a single nil for all, I meant having a null per domain would force people to think of what it means to have nothing in that field and handle it. Maybe I'm too naive.

Sometimes you don't want to allow a value to be null at all, but with null references you can't represent that at the language level.

But for numbers, a zero is not considered null, because it was handled in the operators rules.

Re: Null References: The Billion Dollar Mistake

#45

Post by jrockway » Sat, Jan 11, 2020, 3:06 PM UTC

tibbe wrote: post #18

jrockway wrote: post #9

Earlier quoted context omitted.

I assume two reasons, efficiency and because an efficient implementation of mutable state would have the same problem. Right now, a single sentinel value makes a pointer null or not null (0x0 is null, everything else is not null). This is exactly how you'd implement a stricter type, like "Maybe". Encoded as a 64-bit integer, "Nothing" would be represented as 0x00000000 and "Just foo" would be represented as 0xfoo. No…

This missed the point. The point of not that you can forget to check the null case. The point is that you can express that sometimes there's no null case.

The "no null" case in traditional languages is just "int" instead of "*int". All values inside an "int" are valid integers.

Certainly it's problematic to use the same language primitive to mean "a pointer" and "this might be empty", but it's what people use them for in every language that has pointers (that I've used anyway).

Re: Null References: The Billion Dollar Mistake

#46

Post by DaiPlusPlus » Sat, Jan 11, 2020, 3:07 PM UTC

seventh-chord wrote: post #38

cm2187 wrote: post #25

Earlier quoted context omitted.

Plus also doesn’t the stack need to be small to fit into the CPU cache?

There is absolutely no requirement from the hardware that the stack be any particular size

On the Windows desktop, the default stack size is 1MB. In IIS-hosted applications the default stack size is reduced to 250KB due to the popularity of the now-outdated programming trope of "one thread per request (per-connection)". On x86 Linux the default stack size is 2MB - which seems generous.

Re: Null References: The Billion Dollar Mistake

#47

Post by jrockway » Sat, Jan 11, 2020, 3:15 PM UTC

anderskaseorg wrote: post #16

jrockway wrote: post #9

Earlier quoted context omitted.

I assume two reasons, efficiency and because an efficient implementation of mutable state would have the same problem. Right now, a single sentinel value makes a pointer null or not null (0x0 is null, everything else is not null). This is exactly how you'd implement a stricter type, like "Maybe". Encoded as a 64-bit integer, "Nothing" would be represented as 0x00000000 and "Just foo" would be represented as 0xfoo. No…

> It's exactly the same as langages with null pointers: Four huge differences: 1. You don’t need to pass around ‘Maybe a’ everywhere. If null isn’t expected as a possible value (which usually it isn’t), you just pass around ‘a’, and when you do use ‘Maybe’ it actually means something. 2. The Haskell compiler can, and does (with -Wall), tell you that your pattern match is non-exhaustive. You don’t need a separate “lin…

> You don’t need to pass around ‘Maybe a’ everywhere.

You don't need to pass pointers around everywhere. Languages with null still have value types that cannot be null.

> You don’t need a separate “linter or whatever”.

Optional compiler flags count as "whatever" to me.

> it’s not undefined behavior like in C that could result in something weird and unpredictable happening later (or earlier!)

C++ doesn't define this, but the OS does (and even has help from the CPU).

Anyway, my TL;DR is that it's easy to have a slow program that passes everything by value, or east to have a fast program that uses pointers or references. Removing the special case of null is meaningless, because you can still have a pointer to 0x1 which is just as bad as 0x0, probably. This goes back to my original answer to the question "why don't more languages get rid of null" which was "it's harder than it looks." I think I'm right about that. If it were easy, everyone would be doing it.

Re: Null References: The Billion Dollar Mistake

#48

Post by giulianob » Sat, Jan 11, 2020, 3:16 PM UTC

DaiPlusPlus wrote: post #17

seventh-chord wrote: post #15

Earlier quoted context omitted.

in languages like c, rust or go, where you can put arbitrary data on the stack, it seems to me as if such issues are less common because you dont have to worry about initializing pointers and allocating memory unless you actually want to put something on the heap. Thus if you make everything a reference in your language its no wonder you run into issues like null-pointers more often

With stack allocation you then encounter problems with object lifetime. Rust solves this problem by binding references to scope, and Go solves this by invisibility changing an allocation to the heap (and uses ref-counting? I think?). I wish C had a feature that would let you allocate something on the stack and then return to the parent stack frame without popping the stack-pointer - that would be handy for self-conta…

Regardless of whether it's on the stack or heap the point still stands. If all your objects are randomly allocated then an array is just references to those objects and will start out null. If you're using value types then your array of objects will never be null (empty instead) and you will benefit from CPU caching the data.

Re: Null References: The Billion Dollar Mistake

#49

Post by [deleted] » Sat, Jan 11, 2020, 3:18 PM UTC

jrockway wrote: post #9

x3ro wrote: post #3

This comes up again and again in one form or the other, yet new languages still seem to be making the same mistake. Of all languages I've touched, Rust seems to be the only one that mostly circumvents this problem. Are there other good examples?

I assume two reasons, efficiency and because an efficient implementation of mutable state would have the same problem. Right now, a single sentinel value makes a pointer null or not null (0x0 is null, everything else is not null). This is exactly how you'd implement a stricter type, like "Maybe". Encoded as a 64-bit integer, "Nothing" would be represented as 0x00000000 and "Just foo" would be represented as 0xfoo. No…

[deleted]

Re: Null References: The Billion Dollar Mistake

#50

Post by andrepd » Sat, Jan 11, 2020, 3:30 PM UTC

JakobProgsch wrote: post #43

auxym wrote: post #20

Earlier quoted context omitted.

I've certainly had some "None" errors in Python. I think the difference comes from dynamic vs static typing. In Python, you sort of get into the habit of "defensive" programming: checking inputs to your function, catching Nones, etc. In java, you tend to rely more on the type system. If it typechecks/compiles, there's a good chance it's OK. That is, until you get a null value that's not handled. That's the root issue…

The first line is why I'm not 100% convinced of the severity of this mistake compared to the alternatives. The problem fundamentally is the use of magic values/numbers to represent the concept of "no value". You don't need explicit language support to have that concept and the bugs it causes. I guess having that as an intrinsic concept in the language makes it more likely that people use it badly. On the other hand d…

No, that's not the "fundamental problem". The fundamental problem is a type system that lies. A "pointer to string" is not actually a pointer to a string, it's a pointer to a string or to nothing. If your api returns a pointer of the latter type, it should signal this by making the return type "maybe-pointer to string" (although it has the same memory representation as "pointer to string"). Then, if the user tries to dereference a maybe-pointer (that is, to use a maybe-pointer as a pointer), the type system can statically catch this and make it a simple type check failure compilation error. The user must first check if it's null through a function that casts a maybe-pointer to a pointer.

Nothing about this precludes the usage of sentinel values.