Live data from Hacker News

Fast linked lists

dygalo.dev

111–120 of 131 posts

Re: Fast linked lists

#111
Linked lists are often the wrong choice because they're rarely performant or efficient when compared to vecs in the real world.

In general, use vecs until you absolutely can't.

Perhaps there are a few uses for LL's in limited circumstances.

Re: Fast linked lists

#112
post #37

> Linked lists are taught as fundamental data structures in programming courses, but they are more commonly encountered in tech interviews than in real-world projects. I beg to disagree. In kernels, drivers, and embedded systems they are very common.

I beg to disagree^2. Tasks, threads, and processes are often structured as rings where there is always a "next" to maintain simplicity of task switching. The overall architecture of resources is modelable as cyclic graphs but implemented as rings, deques, single LLs, and other data structures.

Re: Fast linked lists

#113
post #37

> Linked lists are taught as fundamental data structures in programming courses, but they are more commonly encountered in tech interviews than in real-world projects. I beg to disagree. In kernels, drivers, and embedded systems they are very common.

Linked lists were heavily used in application software before the appearance of standard libraries and Java, which is when dynamically sizable array-based lists become common. There also wasn't a gap between the performance of linked lists and arrays before CPU became significantly faster than RAM.

Modern processor and cache performance lend themselves to vectors and SSA. Linked lists just don't scale well outside of niche uses.

Re: Fast linked lists

#114
post #59
post #49

Earlier quoted context omitted.

1%

More like 0.01% -- if we consider enterprise programmers, web programmers, and application/game programmers which I'd expect to be the largest groups...

Yep. There aren't many software developers I know who have ever touched {Linux, macOS, FreeBSD, Windows} kernel code except for embedded devs, driver devs, security researchers, hobbyists, and SREs/PEs.

The % who have touched kernel bits, wrote a triangle engine scene renderer, wrote a compiler, touched server metal in production, have worked on ASICs, and can put together ML/AI building blocks shrinks way, way down to a handful of living humans.

Re: Fast linked lists

#115
post #45

Earlier quoted context omitted.

Most people who take data structures courses or perform tech interviews don't end up working on kernels, drivers, or embedded systems though. To me, it sounds like the point being made is that there are a large number of programmers who have learned about linked lists but haven't run into many cases where they needed them in the world world, and I think it's accurate.

You need a linked list to write hello world in any Lisp, though. Seems like the glaring exception to the rule!

No, you don't need to use linked lists to send a string to the standard output port in most Lisps. You just call a function.

Re: Fast linked lists

#116

Earlier quoted context omitted.

I do want to say that I think the Itanic would have fared way, way better in a post-LLVM world where the importance of smart, optimizing compilers is much more valued and understood and language designers actively work hand-in-hand with compiler devs far more often (with much more significant back-and-forth from hardware manufacturers).

I don't think LLVM is particularly good at optimizing VLIW code. Very good optimizing compilers existed before LLVM. Intel had one specifically for Itanium. It wasn't enough.

Why would llvm be particularly good at optimizing vliw code when there’s no demand for it to be? You can’t believe everything else would remain the same in the hypothetical I posed.

Re: Fast linked lists

#117
post #95
post #14

It strikes me the bottleneck for this problem isn't Vec or List, it's the serde_json Value type that needs to be used. This is useful for serializing/deserializing values into Rust types but if you're trying to validate JSON against a schema you don't actually need the JSON value data, just the types of the nodes (or more specifically, you only need some of the value data, and probably not much of it, so don't pay fo…

I've seen a lexer/parser scheme that encodes the lexer token type along with the token file location information into a u64 integer, something like struct Token { token_type: u8, type_info: u8, token_start: u32, // offset into the source file. token_len: u16 } It's blazing fast. The lexer/parser can process millions of lines per second. The textual information is included, and the location information is included.

I think the key insight is that the true benchmark is bytes/second (bandwidth) of the lexer/parser, so reducing the size of the output data (tokens/AST nodes) is a massive gain in the amount of data that you can process in the same amount of time.

The fewer bytes you can pack data into the more data that you can process per second. Computers may be the most complex machines ever built but the simple fact of having fewer things to touch means you can touch more things in the same amount of time remains true.

Re: Fast linked lists

#119

Earlier quoted context omitted.

I don't think LLVM is particularly good at optimizing VLIW code. Very good optimizing compilers existed before LLVM. Intel had one specifically for Itanium. It wasn't enough.

Why would llvm be particularly good at optimizing vliw code when there’s no demand for it to be? You can’t believe everything else would remain the same in the hypothetical I posed.

A) optimizing for VLIW is hard. B) the null hypothesis would be no change.

Re: Fast linked lists

#120
post #45

Earlier quoted context omitted.

Most people who take data structures courses or perform tech interviews don't end up working on kernels, drivers, or embedded systems though. To me, it sounds like the point being made is that there are a large number of programmers who have learned about linked lists but haven't run into many cases where they needed them in the world world, and I think it's accurate.

You need a linked list to write hello world in any Lisp, though. Seems like the glaring exception to the rule!

As source code, but not necessarily as running code.

SBCL:

    * (defun hello-world () (write-string "hello world"))
    HELLO-WORLD

    * (disassemble #'hello-world)
    ; disassembly for HELLO-WORLD
    ; Size: 36 bytes. Origin: #x100311C85C                        ; HELLO-WORLD
    ; 5C:       AA0A40F9         LDR R0, [THREAD, #16]            ; binding-stack-pointer
    ; 60:       4A0B00F9         STR R0, [CFP, #16]
    ; 64:       EAFDFF58         LDR R0, #x100311C820             ; "hello world"
    ; 68:       570080D2         MOVZ NARGS, #2
    ; 6C:       29EC80D2         MOVZ TMP, #1889
    ; 70:       BE6B69F8         LDR LR, [NULL, TMP]              ; WRITE-STRING
    ; 74:       DE130091         ADD LR, LR, #4
    ; 78:       C0031FD6         BR LR
    ; 7C:       E00120D4         BRK #15                          ; Invalid argument count trap
The actual code for this example is machine code (which references a string, which is a vector), here without linked lists.
Post reply on HN