Speed of Rust vs. C

I didn't say anything about arena allocators. What I said was that amortizing allocation was routine and commonplace in ripgrep's code. I definitely wouldn't say that amortizing allocation is "special" or something I use a "few" times. As one example of amortizing allocs, it's very common to ask the caller for some memory instead of allocating memory yourself.

> I am pretty tired of the "you don't understand Rust, therefore you are beneath us" line that everyone in the Rust community seems to deploy with even the slightest provocation

Kind of like opening a comment with "This entire article is nonsense." Right? Snubbing your nose and then getting miffed by the perception of others snubbing their nose at you is a bunch of shenanigans. And then you snub your nose at pretty much everyone: "It's the blind leading the blind." I mean, c'mon dude.

The problem with your comments is that they lack specifics. Even after this comment where you've tried to explain, it's pretty hard for me to understand what you're getting at. I suspect part of the problem is your use of this term "bulk allocation." Is it jargon that refers to the specific pattern you have in mind? Because if it is, I can't find it documented anywhere after a quick search. If it's not jargon, then "bulk allocation" could mean a lot of things, but you clearly have a very specific variant of it in mind.

It's clear to me that your argument is a very subtle one that requires nuance and probably lots of code examples to get the point across. Going about this at the other end---with lots of generalities and presumptions---just seems like a fool's errand.

> > For example, in C I'd be tempted to reuse a buffer allocated for one purpose for another purpose later (a technique known as HEARTBLEED).

> You can do that in Java (with byte arrays) or in Common Lisp, so what is the point here? It is not practice in Java, Lisp nor in C and C++.

C is a really old language with ancient libraries that are still widely used even though they are simply bad by modern standards. For that reason, I roll my eyes when people say something is not practice in C or talk about "sane" C libraries. A big part of working with C is dealing with ancient insanity.

You can make much stronger statements about what is idiomatic in Rust (and to some extent Java) simply because it's newer and more cohesive.

> > It's convenient to have fixed-size buffers for variable-size data (e.g. PATH_MAX) to avoid (re)allocation of growing buffers

> This is because OS/Kernel/filesystem guarantee path max size.

I think you've got that backwards. There's an advertised max path size because people wanted to stick paths in fixed-size buffers rather than deal with dynamic allocation. PATH_MAX is fairly arbitrary considering that there are certainly ways of creating and opening files which have paths exceeding that limit. I found this doc talking about this: https://eklitzke.org/path-max-is-tricky

> printf is not shipped with the OS, but with libc runtime.

"The OS" doesn't mean "the kernel". Read...anything...even the lackluster wikipedia article about operating systems...and you'll see stuff like GUIs described as part of the OS. They (generally) don't mean those are in the kernel. You can also see this for example in the GNU GPL; they call out "system libraries", which certainly includes libc.

> So you are saying Rust doesn't call (g)libc at all and directly invoke kernel interrupts? Sure, you can avoid this print "overhead" in C with 3-4 lines of inline assembly, but, why?

Rust's own standard library uses libc's system call wrappers but not stdio. It has its own libraries for buffer management and formatting which provide the safety one would expect of Rust, know how to integrate with Rust's Display trait for formatting arbitrary Rust data structures, etc. You could call libc::printf yourself if you wanted to, but that's not idiomatic. I wrote some Rust code calling libc::vsnprintf just the other day, because I got a format string + va_list from C in a log callback.

Ah! I think I am understanding you a bit better. The thing is, ultimately, Rust is as flexible as you want it to be, and so there are a variety of options. This can make it tricky, when folks are talking about slightly different things, in slightly different contexts.

When you say "doesn't reach out to user level," what I mean by what I said was that users don't generally call alloc and dealloc directly. Here, let's move to an actual concrete example so that it's more clear. Code is better than words, often:

    use bumpalo::{Bump, boxed::Box};

    struct Point {
        x: i32,
        y: i32,
    }

    fn main() {
        let bump = Bump::with_capacity(256);

        let c = Box::new_in(Point { x: 5, y: 6 }, &bump);
    }

But notably, I am not using any unsafe here. Yes, I am saying "give me an allocation of this total size", and yes I am saying "please allocate stuff into it and give me pointers to it," but generally, I as a user don't need to mess with unsafe unless I'm the person implementing bumpalo. And sometimes you are! Personally, I work in embedded, with no global heap at all. I end up using more unsafe than most. But there's no unsafe code in what I've written above, but it's still gonna give you something like what you said you're doing in your current game. Of course, you probably want something more like an arena, than a pure bump allocator. Those exist too. You write 'em up like you would anything else. Rust will still make sure that c doesn't outlive bump, but it'll do that entirely at compile time, no runtime checks here.

Oh, and this is sorta random but I didn't know where to put it: Rust's &str type is a "pointer + length" as well. Using this kind of thing is extremely common in Rust, we call them "slices" and they're not just for strings.

> You can say, "well you as the end-user shouldn't be doing this stuff, everything should be wrapped in structures that were written by someone smarter than you I guess," but that is just not the model of programming that I am doing.

While that's convenient, and in this case, I am showing that, the point is that it's about encapsulation. I don't have to use this existing allocator if I wanted to write something different. But because I can encapsulate the unsafe bit, no matter who is writing it, I need to pay attention in a smaller part of my program. Maybe I am that person, maybe someone else is, but the benefit is roughly the same either way.

> So if you add back in worrying about lifetimes, it's not the same thing.

To be super clear about it, Rust has raw pointers, that are the same as C. No lifetimes. If you want to use them, you can. The vast, vast, vast majority of the time, you do not need the flexibility, and so it's worth giving it up for the compile time checks.

> If you think "bulk memory allocation" is like...

It's not clear to me above if the API I'm talking about above is what you mean here, or something else. It's not clear to me how you'd get simpler than "please give me a handle to this part of the heap," but I haven't seen your latest Jai streams. I am excited to give it a try once I am able to.

> but using bulk allocation broadly and freely across your program goes against the core spirit of the language

I don't know why you'd think these techniques are against the core spirit of the language. Rust's primitive array type is literally "give me N of these bits of data laid out next to each other in memory." We had a keynote at Rustconf about how useful generational arenas are as a technique in Rust. As a systems language, Rust needs to give you the freedom to do literally anything and everything possible.

> One last note, I am pretty tired of the "you don't understand Rust, therefore you are beneath us"

To be clear, I don't think that you or anyone else is "beneath us," here. What I want is informed criticism, rather than sweeping, incorrect statements that lead people to believe things that aren't true. Rust is not perfect. There are tons of things we could do better. But that doesn't mean that it's not right to point out when facts are different than the things that are said. You of all people seem to appreciate a forward communication style.