Live data from Hacker News

John Carmack on Functional Programming in C++ (2018)

sevangelatos.com

161–170 of 179 posts

Re: John Carmack on Functional Programming in C++ (2018)

#161
post #158

Earlier quoted context omitted.

> [1] https://github.com/louthy/language-ext Cool library. I've had a few of these patterns in my Sasa library for years, but you've taken it to the Haskell extreme! Probably further than most C# developers could stomach. ;-) You might be interested in checking out the hash array mapped trie from Sasa [1]. It cleverly exploits the CLR's reified generics to unbox the trie at various levels which ends up saving quite a…

> so it performs almost on par with the mutable dictionary. I notice you have a comment that says "This trie is now the fastest immutable dictionary I'm aware of". Unfortunately I have to make you aware that my implementation is faster, sorry! ;) Here's the add-items benchmark: BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1265/22H2/2022Update/SunValley2) AMD Ryzen Threadripper PRO 3995WX 64-Cores, 1 CPU, 128 lo…

Nice! I'm away for the weekend so will follow-up with an explanation of the nested generic when not on the phone if the following is unclear: the basic idea is that you inline the next level of the Trie into the previous level, thus collapsing the depth and removing some indirections. This slightly bloats the internal tree nodes but the leaves vastly outnumber the internal nodes, and since the leaves have been inlined one level up, it ends up compacting it somewhat. This is maybe clearer when starting with a struct-only design for the internal nodes and then performing the inlining as an optimization on that.

Also wondering whether you checked the Trie from the latest repo or the older struct version I also linked. I think the struct version was a little faster but I switched for the aforementioned idiomatic reasons.

I'll have to check out the CHAMP design for sure, I'm curious how it differs.

Edit: I also recall that I didn't optimize iteration at all and that it was fairly inefficient, but can't really confirm now. I recall optimizing merge carefully though so that should perform well.

Edit 2: forgot to mention that Node generic is nested 6 times because that's the max depth of the trie for 32 bit keys with 32 element nodes.

Re: John Carmack on Functional Programming in C++ (2018)

#162
post #99

Earlier quoted context omitted.

> And how do you know you've constricted your data enough? Types are composable. Sum-types and product-types allow the composition of smaller types into larger ones. They come in the form of discriminated-unions and records in FP languages. So when it comes to the question of how do I know when I've constricted my data enough, it's when I know all the components of all types have been constricted enough. You don't ha…

>I know when I've constricted my data enough, it's when I know all the components of all types have been constricted enough. Do you not see that this is a tautology? >That's our job, to translate the requirements into code, and the logic is that bit and therefore more prone to human error. The requirements given are typically ambiguous and it is our job to come up with the complete specification ourself. Our specific…

>I know when I've constricted my data enough, it's when I know all the components of all types have been constricted enough. > Do you not see that this is a tautology?

It might be if you hadn't butchered what I wrote. If I have a type called Month, and it can only accept values 1-12. Then I know I have constrained that type enough. If I then create types called Day and Year and constrain those I know they're constrained enough.

If I then compose Day, Month, Year into a type called Date and check the rules of the number of Days in a month so that an invalid date can't be instantiated then I have a more complex type leveraging the simpler ones. I could then put Date into a record type called Appointment, etc. etc. For each type I create I know what data I am trying to represent, so I constrain the type at that point. There's no tautology here, it's just composition of types. Making larger types from smaller ones and making sure they can't ever hold bad state.

> Inheritance is just a different way to create sum types.

Not really, they're open. Sum-types are closed. i.e. they represent a concrete set of states. The openness of an inheritance hierarchy is the problem. We nearly never need completely open type hierarchies, it is pretty rare that something needs to be extensible in that way outside of library development.

That doesn't mean inheritance is always bad, but the 'always inheritance' approach that is often championed in the OO world certainly is.

The goal is to write total functions [1] where every value in the domain can be mapped to a value in the co-domain. If that can't happen then your code is less predictable (throws exceptions or has undeclared side-effects). Having complete control of the states of any data-type is critical to making the approach easy to implement.

[1] https://www.statisticshowto.com/total-function/

Re: John Carmack on Functional Programming in C++ (2018)

#163
post #158

Earlier quoted context omitted.

> so it performs almost on par with the mutable dictionary. I notice you have a comment that says "This trie is now the fastest immutable dictionary I'm aware of". Unfortunately I have to make you aware that my implementation is faster, sorry! ;) Here's the add-items benchmark: BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1265/22H2/2022Update/SunValley2) AMD Ryzen Threadripper PRO 3995WX 64-Cores, 1 CPU, 128 lo…

Nice! I'm away for the weekend so will follow-up with an explanation of the nested generic when not on the phone if the following is unclear: the basic idea is that you inline the next level of the Trie into the previous level, thus collapsing the depth and removing some indirections. This slightly bloats the internal tree nodes but the leaves vastly outnumber the internal nodes, and since the leaves have been inline…

> Also wondering whether you checked the Trie from the latest repo or the older struct version I also linked

I included Sasa.Collections 1.0.0-RC4 from Nu-Get. So I assume it's your latest version. I've added Sasa.Collections permanently to my benchmarks seeing as you're the first library to get close to the performance of the language-ext collections - I'd like to keep an eye on you, haha! Always good to have a bit of friendly competition :)

> forgot to mention that Node generic is nested 6 times because that's the max depth of the trie for 32 bit keys with 32 element nodes.

Yep, I figured that one. Still I look at it and go 'huh?'. I've been looking at code all day though, so I'll have a proper look tomorrow when less tired.

Re: John Carmack on Functional Programming in C++ (2018)

#164
post #163

Earlier quoted context omitted.

Nice! I'm away for the weekend so will follow-up with an explanation of the nested generic when not on the phone if the following is unclear: the basic idea is that you inline the next level of the Trie into the previous level, thus collapsing the depth and removing some indirections. This slightly bloats the internal tree nodes but the leaves vastly outnumber the internal nodes, and since the leaves have been inline…

> Also wondering whether you checked the Trie from the latest repo or the older struct version I also linked I included Sasa.Collections 1.0.0-RC4 from Nu-Get. So I assume it's your latest version. I've added Sasa.Collections permanently to my benchmarks seeing as you're the first library to get close to the performance of the language-ext collections - I'd like to keep an eye on you, haha! Always good to have a bit…

Start with this representation:

    struct Node
    {
      uint bitmap;
      KeyValuePair[] entries;
      Node[] children;
      // Node = Leaf of bitmap * children | Internal of bitmap * children
      // We inline that sum into a struct where
      // Leaf -> entries != null
      // Internal -> children != null
    }
Now consider the case where leaves are somewhat sparse. That means the children arrays will have few entries, so instead of allocating a bunch of one-element arrays for children which themselves allocate a bunch of one-element arrays of values, just inline the value arrays in the parent. Now propagate that same idea for internal nodes all the way back to the root.

Also, because these are all structs, the JIT should specialize the code generated for each level so it should be pretty fast. IIRC, the speed difference between the original and the inlined representation was noticeable but not crazy, but the memory use difference was considerable. This was all years ago though, first implementation was 2013.

Re: John Carmack on Functional Programming in C++ (2018)

#165

Earlier quoted context omitted.

C99 with code generators written in python or lua for the engine. Implementing templates in the C preprocessor goes pretty badly, but implementing them as a normal language that writes valid C to #include various places works just fine.

> C99 with code generators written in python or lua for the engine. Nice fantasy, but the real-life codebases that do this make even the most hellish C++ code look like future space-age magic technology. You're not speaking from real-world pragmatic (heh) experience here, you're just complaining that C++ is hard and that you'd rather go shopping.

I've been coding like that since at least 2016[1]. The trick is to make adding code generators as easy as adding source code. Not a game engine in my case, though there's a CppCon talk from 2014 which makes a passing reference to using code generators with C++ instead of templates for one.

Assume source is src/foo.c and compiled to obj/foo.o, then introduce a third directory called gen. Change the makefile rule to copy .c from src to gen, and compile from gen to obj. Then add a makefile rule that says gen/foo.c can be built from src/foo.c.py by calling python on the source and redirecting stout.

That means a C source file can be turned into a python program that generates the same source by wrapping it in a string literal, calling print and renaming the source file. No build system change. Then change the python as you see fit to work around the limitations of C.

Works really well for a solo developer. Repo using this scheme is ~100 generated files (mostly lua, some python) vs ~600 C files. Plus ~20 C++ and one D, just getting started with that language.

[1] https://github.com/JonChesterfield/boring-makefile/blob/mast...

  $(GEN_PY_SOURCE): $(GEN_ROOT_DIR)%.c: $(SRC_ROOT_DIR)%.c.py
   $(PYTHON) $^ > $@

Re: John Carmack on Functional Programming in C++ (2018)

#166
post #162

Earlier quoted context omitted.

>I know when I've constricted my data enough, it's when I know all the components of all types have been constricted enough. Do you not see that this is a tautology? >That's our job, to translate the requirements into code, and the logic is that bit and therefore more prone to human error. The requirements given are typically ambiguous and it is our job to come up with the complete specification ourself. Our specific…

>I know when I've constricted my data enough, it's when I know all the components of all types have been constricted enough. > Do you not see that this is a tautology? It might be if you hadn't butchered what I wrote. If I have a type called Month, and it can only accept values 1-12. Then I know I have constrained that type enough. If I then create types called Day and Year and constrain those I know they're constrai…

>into a record type called Appointment, etc. etc

This breaks down in the etc. What if an appointment is not valid on them weekend, or a holiday, or when someone's child has soccer practice, or when that appointment conflicts with another one. There are all sorts of restriction that can be added and yes they technically cover some potential bug that someone technically could introduce. It may be hard to predict all of these restrictions ahead of time.

>Not really, they're open

At compile time 99% of the time they are closed. Languages also add features to let you make it closed.

>If that can't happen then your code is less predictable (throws exceptions or has undeclared side-effects).

This is impossible in the real world where hardware issues are real. Exceptions and crashes are okay and is just a fact of life when working with large software syshems. We know these systems are not going to be perfect and embrace it rather than trying to perfectly handle every single thing perfectly every time. It's accepting that these bugs exist and that processes should be put in place in identifying them and monitoring them so they can be fixed.

Total functions also don't allow for infinite loops which is common place in real systems that mare expected to potentially run forever instead of only being able to serve 1000 requests before exiting.

Re: John Carmack on Functional Programming in C++ (2018)

#167
I’ve considered a lint rule that requires all functions to be written in Lambda syntax so that closures, side effects, and purity can be explicit. However, I haven’t explored possible effects on compile time, size, and performance. Another possibility is explicit scope closures, like [x,&y]{…} available for every defined scope.

Re: John Carmack on Functional Programming in C++ (2018)

#168

Earlier quoted context omitted.

It took me awhile to realize but this is one of the big things that algebraic data types (ADTs) help you to do...design your data types so they have exactly the number of valid states that your domain has. To use another common FP way of saying it...make invalid states unrepresentable.

It's not just the ADTs but the exhaustive pattern matching that help make sure you've covered all of the cases in your ADT. You'll see JavaScripters use objects as a poor man's ADT where they'll use the key name as a the constructor or a { tag: key, ... }, but that has caveats (aside from feeling less first-class): 1) JavaScript can't do exhaustive pattern matching so you'll always need to handle null cases, 2) check…

That Typescript ADT example is heavily outdated (or explicitly obtuse)

class Bar { tag="bar" as const; constructor(public value:string){} } class Baz { tag="baz" as const; constructor(public value:boolean){} } type Foo = Bar|Baz;

Re: John Carmack on Functional Programming in C++ (2018)

#169
post #37
post #28

Earlier quoted context omitted.

I agree, but most would access internal state, which is equally impure. A pure function has no access to anything outside of its function parameters, and it changes nothing outside the scope of the function. In the context of a pure function, a local member variable is no different than a global variable because it’s outside the scope of the actual function. If you can’t take the function away from the class and have…

why do you think the length member function is not pure?

Member functions have an invisible argument: the pointer to the C++ object. You can access this argument using the 'this' keyword. John Carmack describes pure functions like so:

> Pure functions have a lot of nice properties. Thread safety. A pure function with value parameters is completely thread safe. With reference or pointer parameters, even if they are const, you do need to be aware of the danger that another thread doing non-pure operations might mutate or free the data, but it is still one of the most powerful tools for writing safe multithreaded code.

Emphasis on 'value parameters'. The invisible 'this' argument is not passed by value, it is passed by reference. By Carmack's definition, C++ member functions are not pure.

Re: John Carmack on Functional Programming in C++ (2018)

#170
post #163

Earlier quoted context omitted.

> Also wondering whether you checked the Trie from the latest repo or the older struct version I also linked I included Sasa.Collections 1.0.0-RC4 from Nu-Get. So I assume it's your latest version. I've added Sasa.Collections permanently to my benchmarks seeing as you're the first library to get close to the performance of the language-ext collections - I'd like to keep an eye on you, haha! Always good to have a bit…

Start with this representation: struct Node { uint bitmap; KeyValuePair [] entries; Node [] children; // Node = Leaf of bitmap * children | Internal of bitmap * children // We inline that sum into a struct where // Leaf -> entries != null // Internal -> children != null } Now consider the case where leaves are somewhat sparse. That means the children arrays will have few entries, so instead of allocating a bunch of o…

I belatedly realized this might require some explanation too. Sorry, too many years of experiments and microoptimizations.

This is an unboxed representation of a sum type. Rather than relying on the CLR's inheritance to model sums I lay it out explicitly at the cost of a little space. It may seem wasteful since it looks like I'm adding 8 unused bytes to each node, but the CLR object header is 16 bytes so you're actually saving 8 bytes per node by avoiding classes and using structs that have no header! You can really see the difference in the working set if you do a full GC.Collect and check the allocated bytes after constructing some large tries.

Also, because I'm removing a heap object per node, this representation incurs one less indirection per node accessed, so I'm doing half as many loads as an equivalent class-based representation for every operation. This is a large constant factor difference, and probably partly explains why Sasa's ContainsKey is faster than yours.

Finally, no trie operatioms incur any virtual dispatch, which are instead replaced by a simple if-checks which are generally more predictable. This probably has less impact these days since branch predictors are pretty good now.

I think if you replaced your class-based representation that uses the superior CHAMP model with this unboxed representation your hash map would be the uncontested champion. You might even challenge the mutable dictionary for performance!

Post reply on HN