Null References: The Billion Dollar Mistake

This old chestnut again.

There is an inherent problem in designing processes and writing code to capture them: The notion of not-a-value.

There are great many ways to solve them. The most common ones are 'null' and 'Optional[T]'. Neither just makes the problem magically go away. If a process is designed (or a programmer writes it) thinking that 'ah, well, here, not-a-value cannot happen', but it can, then.. you have a bug.

Some language features might make it possible to help reduce how often it occurs, but eliminate it? I don't think so.

Imagine, for example, in an Optional based language, that you just map the optional to a lambda to execute on the optional, and the behaviour of the optional is to then simply silently do nothing if it's optional.none. That'd be a much harder to find bug than a nullpointer error. (errors with stack traces pointing at the problem are obviously vastly superior to mysterious do-nothing behaviour with no logs or traces of any sort!).

Some other creative solutions:

* [Pony](https://www.ponylang.io/) tries to be very careful about registering when an object is 'valid' and when it isn't, and when you write code, you have to say which state the objects you interact with can be in. This lets you avoid a lot of the issues... but pony is quite experimental.

* In java you can annotate any usage of a type with nullity info, and then compiler linter tools will simply tell you that you have failed to take into account a potential null value. You are then free to ignore these warnings if you're just writing test code, or know better. Avoids clogging up the works with optional, but as the java ecosystem shows, you can't just snap your fingers and make 30 years of massive community effort instantaneously instantly be festooned with 'might-not-hold-a-value' style information. At least the annotation style gives the hope of being backwards compatible (to be clear, optional, for java? Really bad idea).

* in ObjC, if you send a message to a null pointer, it silently does nothing, in contrast to virtually all other languages with null types where attempting to message a null ref causes an error or even a core dump.

* Just write better APIs. Have objects that represent blank state (empty strings, empty collections, perhaps dummy streams which provide no bytes / elements, etc). For example, in java: Java's map (a dictionary implementation) has the `.get(key)` method which returns the value associated with that key, and returns `null` if there is no such value. About 6 years ago another method was added in a backwards compatible fashion (so, all java map implementations got this automatically): `getOrDefault(key, defaultValue)`. This one returns the provided default value if key isn't in the map. You'd think optionals provide a general mechanism for this, but, in scala, you have both: There's `someMap get(key)` which returns an optional, so to get the 'give me a default value' behaviour, that'd be `someMap.get(key).getOrElse(defaultValue)`, but maps in scala also have the java shortcut: `someMap.getOrElse(key, defaultValue)`. Sufficient thought in your APIs mostly obviates the issues.

null is not a milion dollar mistake. It is a solution to an intrinsic problem with advantages and disadvantages over other solutions.

I remember tracking down the null silent message failure issue in the early 1990s on NextStep. Then again almost 2 decades later on the iPhone. Personally, I’m not a fan of silent failures.

IMHO, allowing for non-nullable variables is a huge improvement in language design. Adding boilerplate annotations is an ugly way to handle it. Optimize for the common case and make variables non-nullable by default.

in languages like c, rust or go, where you can put arbitrary data on the stack, it seems to me as if such issues are less common because you dont have to worry about initializing pointers and allocating memory unless you actually want to put something on the heap. Thus if you make everything a reference in your language its no wonder you run into issues like null-pointers more often

With stack allocation you then encounter problems with object lifetime. Rust solves this problem by binding references to scope, and Go solves this by invisibility changing an allocation to the heap (and uses ref-counting? I think?).

I wish C had a feature that would let you allocate something on the stack and then return to the parent stack frame without popping the stack-pointer - that would be handy for self-contained object-constructors.

The mistake is being nullable/optional by "default", that is with the least amount of effort for programmers using such a language. Or worse only ever nullable (like Java is except for its built-in scalars I think?).

There is obviously a need about having optional things, but this is not the common case, so this should not be the default and even less the only solution. And it should enforce handling the absent case.

"null" is a shortcut for talking about solution which does nothing of that (and is even UB in case of mistake in some languages). Billion Dollar Mistake is generously low; probably the cost is already Multi-Billion Dollar, and counting.