Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes by djb on using Fil-C

cr.yp.to

Post reply on HN
Print view View original

171–180 of 263 posts

Re: Notes by djb on using Fil-C

#171

Post by [deleted] »

simonask wrote: post #69
tkz1312 wrote: post #52

Earlier quoted context omitted.

I do not think this is niche in the slightest. I would very happily take a 2-4x slowdown for almost all of the web facing C software I run if I get guaranteed memory safety. I will be using at the very least fil-c openssh (and likely much more) on every machine I run.

Sure, that makes sense. The point I’m making is just that from an engineering perspective, that also implies that there is no longer any reason for that software you’re running to be written in C at all.

[deleted]

Re: Notes by djb on using Fil-C

#172

Post by brucehoult »

collinfunk wrote: post #140
mbrock wrote: post #53

Earlier quoted context omitted.

I think that's the build of LLVM+Clang itself.

Yes, linking LLVM takes up a lot of memory. The documented guidance is to allow one link job per 15 GB of RAM [1]. [1] https://llvm.org/docs/CMake.html#frequently-used-llvm-relate...

And, fairly uniquely, LLVM has a LLVM_PARALLEL_LINK_JOBS setting that is distinct from the number of parallel jobs for everything else. I think I was using that 15 years ago.

I wish GCC had it. I have a quad core machine with 16 GB RAM that OOMs on building recent GCC -- 15 and HEAD for sure, can't remember whether 14 is affected. Enabling even 1 GB of swap makes it work. The culprit is four parallel link jobs needing ~4 GB each.

There are only four of them, so a -j8 build (e.g., with HT) is no worse.

Re: Notes by djb on using Fil-C

#173

Post by brucehoult »

simonask wrote: post #75
mbrock wrote: post #46

Earlier quoted context omitted.

I don't understand the need to hammer in the point that Fil-C is only valuable for this tiny, teeny, irrelevant microscopic niche, while not even talking about what the niche is? To be clear, the niche is rebuilding your entire GNU/Linux userland with full memory safety and completely acceptable performance, tomorrow, without rewriting anything, right? Is this such a silly little idiosyncratic hobby?

So I don’t want to come off as dismissive of the effort - it’s certainly impressive! The reason I’m not super excited is based on the widely publicized findings from Google and Microsoft (IIRC) about memory safety issues in their code: The vast majority is in new code. As such, the returns on running the entire userspace with Fil-C may be quite diminished from the get-go. Those who need to guard against UB bugs in se…

> The reason I’m not super excited is based on the widely publicized findings from Google and Microsoft (IIRC) about memory safety issues in their code: The vast majority is in new code

This makes perfect sense to me.

Which is why I don't at all understand the current fetish with rewriting things that have been working well for decades in Rust. Such as coreutils. Or apt.

It feels like an almost deliberate crippling of progress by diverting top talent into useless avenues, much like string theory in physics, or SLS/Artemis.

Re: Notes by djb on using Fil-C

#174

Post by skissane »

pizlonator wrote: post #107
kragen wrote: post #57

Earlier quoted context omitted.

I think Fil-C is for people who are using software that has already been written, not for people who are trying to pick what language to write new software in. A substantial amount of software has, after all, already been written.

It's super fun to write C and C++ code in Fil-C because it's like this otherworldly crossover between Java and C/C++: - Unlike Java, you get fantastic startup times. - Unlike Java, you get access to actual syscall APIs. - Unlike Java, you can leverage the ecosystem of C/C++ libraries without having to write JNI wrappers (though you do have to be able to compile those libraries with Fil-C). - Like Java, you can just `…

> Like Java, you can just `new` or `malloc` without `delete`ing or `free`ing.

Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs?

Can the GC be configured to warn or panic if something is GCed without free()? Then you could detect memory leak bugs by recompiling with Fil-C - with less overhead than valgrind, although I’m guessing still more than ASan - but more choices is always a good thing.

Re: Notes by djb on using Fil-C

#175

Post by bangaladore »

GTP wrote: post #49
simonask wrote: post #44

Earlier quoted context omitted.

If that's all you need, the state of the art is very available already through the JVM and the .NET CLR, as well as a handful others depending on your use case. Most of those also come with decent languages, and great facilities to leverage the GC to its maximum. But GCs aren't magic and you will never get rid of all the overhead. Even if the CPU time is not noticeable in your use case, the memory usage fundamentally…

I would say that Rust would be a better choice rarher than patching memory safety on top of C. But I think the reason for this is that most, if not all, cryptographic reference implementations are in C. So they want to use existing reference implementations without having to port them to Rust. IMO cryptographers should start using Rust for their reference implementations, but I also get that they'd rather spend their…

Memory safety is a very small concern for most cryptographic implementations (e.g Side Channel attacks). Rust solves essentially none of the other concerns.

Re: Notes by djb on using Fil-C

#176

Post by brucehoult »

mbrock wrote: post #101
petesergeant wrote: post #97

Earlier quoted context omitted.

> Fil-C is new and is a viable competitor to rust I’ve no horse in the race here, but the Fil-C page talks about a 4x overhead from using it, which feels like it would make it less competitive

Currently measured worst case for some types or code.

I tried it on my primes micro-benchmark (http://hoult.org/primes.txt) and got a 2:1 slowdown on 13th gen i9.

It does a LOT of array access and updating, probably near to worst-case for code that isn't just a loop copying bytes.

The average slowdown is probably more in the same region as using Java or C# or for that matter C++ std::array or std:vector.

Re: Notes by djb on using Fil-C

#177

Post by brucehoult »

testdelacc1 wrote: post #144
vorador wrote: post #110

Earlier quoted context omitted.

No, back in the day C was used for everything. Vim was not written in C because it needed to wring every last bit of performance out of text editing. Rewriting everything in rust "for memory-safety" is a false tradeoff given the millions of lines of C code out there and the fact that rewrites always introduce new bugs.

Please, I’m begging you, stop talking about Rust. You’re shoehorning Rust into a discussion where it hasn’t been mentioned, just to hate on some imaginary people you think are pushing Rust here. No one is talking about that. You sound deranged and obsessed. The vast majority of the conversation here is about GC and the performance implications of that. Please stick to the rest of the thread.

I almost always find that building Boehm GC as a malloc replacement (malloc() -> GC_malloc(), free() -> NOP), and then using LD_PRELOAD to get it used makes any random C/C++ program not only still work but also run faster.

Not only that, but you can then use GC_FREE_SPACE_DIVISOR to tune RAM usage vs speed to your liking on a program by program (or even instance by instance) basis, something completely impossible with malloc().

Re: Notes by djb on using Fil-C

#178

Post by Findecanor »

kragen wrote: post #24
commandersaki wrote: post #23

Earlier quoted context omitted.

I guess to get on board with this, it is my understanding you have to accept the premise of a Garbage Collector in the runtime?

So far we haven't found a viable alternative; CHERI has holes in its temporal integrity guarantees.

Both Fil-C and CHERI rely on a concurrent GC/a GC-like task to find and invalidate all pointers to free()'d memory objects (in "quarantine") before putting them back into the memory pool.

The difference is that because Fil-C has bounds in each object's header, it only has to nullify it to remove access whereas in CHERI a quarantined object can still be accessed through any pointer that hasn't been invalidated yet.

I've seen discussions on adding an additional memory tag to CHERI for memory in quarantine, but I dunno what is best.

Fil-C relies on the compiler being trusted whereas CHERI does not. If we do, then perhaps we could come up with a hardware-accelerated system that is more lightweight than either.

Re: Notes by djb on using Fil-C

#179

Post by kragen »

quotemstr wrote: post #162
silotis wrote: post #152

Earlier quoted context omitted.

This depends heavily on what problem domain you're talking about. For example, a DBMS is necessarily going to shuffle a lot of data into and out of memory.

It depends. Consider DuckDB or another heavily vectorized columnar DB: there's a big part of the system (SQL parser, storage chunk manager, etc.) that's not especially performance-sensitive and a set of tiny, fast kernels that do things like predicate-push-down-based full table scans, ART lookups, and hash table creation for merge joins. DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI. But you…

> DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI.

While I'm not a big fan of rewriting things, all of DuckDB has been written in the last 10 years. Surely a rewrite with the benefit of hindsight could reach equivalent functionality in less than 10 years?

Re: Notes by djb on using Fil-C

#180

Post by kragen »

loeg wrote: post #121
jitl wrote: post #83

Earlier quoted context omitted.

stuff that talks to "the internet" and runs as "root" seems like a good thing to build with filc.

It probably uses OS sandboxing primitives already.

In normal operation, apt has to be able to upgrade the kernel, the bootloader, and libc, so it can't usefully be sandboxed except for testing or chroots.
Post reply on HN