Is there a reason that some of the linked benchmarks, if I'm reading it right, have Fil-C running faster than C?[0] I assume it's just due to micro-benchmark variability but I'm curious. Some of them seem impossibly fast compared to C so I wonder if there are some correctness issue there.
Notes by djb on using Fil-C
181–190 of 263 posts
Re: Notes by djb on using Fil-C
#182Earlier quoted context omitted.
So I don’t want to come off as dismissive of the effort - it’s certainly impressive! The reason I’m not super excited is based on the widely publicized findings from Google and Microsoft (IIRC) about memory safety issues in their code: The vast majority is in new code. As such, the returns on running the entire userspace with Fil-C may be quite diminished from the get-go. Those who need to guard against UB bugs in se…
Hmm, so if they're writing new memory unsafe code in C/C++, presumably to remain within their already established and entrenched C/C++ ecosystems, why isn't Fil-C interesting as a way to thwart memory safety issues in that new code?
Because every problem detected by Fil-C is already a serious problem in the existing code.
As a mitigation strategy, that becomes less interesting as the quality of that code increases, but you still pay the full cost regardless of whether there are actually any bugs.
That can certainly be valuable to you, but as a developer, the more interesting proposition is about how not to ship bugs in the first place.
Re: Notes by djb on using Fil-C
#183Earlier quoted context omitted.
It's super fun to write C and C++ code in Fil-C because it's like this otherworldly crossover between Java and C/C++: - Unlike Java, you get fantastic startup times. - Unlike Java, you get access to actual syscall APIs. - Unlike Java, you can leverage the ecosystem of C/C++ libraries without having to write JNI wrappers (though you do have to be able to compile those libraries with Fil-C). - Like Java, you can just `…
> Like Java, you can just `new` or `malloc` without `delete`ing or `free`ing. Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Can the GC be configured to warn or panic if something is GCed without free()? Then you could detect memory leak bugs by recompiling with Fil-C - with less overhead than valgrind, although I’m guessing still…
Yeah, it panics when you use after free. [1]
But I'm not sure it's worth porting your code to Fil-C just to get that property. Because Fil-C still needs to track the memory allocation with its garbage collector. there isn't much advantage to even calling free. If you don't have a use-after-free bug, then it's just adding the overhead of marking the allocation as freed.
And if you do have a use-after-free bug, you might be better off just letting it silently succeed, as it would in any other garbage collected language. (Though, probably still wise to free when the data in the allocation is now invalid).
IMO, if you plan to use Fil-C in production, then might as well lean on the garbage collector. If you just want memory safety checking during QA, I suspect you are better off sticking with ASan. Though, I will note that Fil-C will do a better job at detecting certain types of memory issues (but not use-after-free)
[1] See the "Use After Free example on: https://fil-c.org/invisicaps_by_example
Re: Notes by djb on using Fil-C
#184Earlier quoted context omitted.
> Like Java, you can just `new` or `malloc` without `delete`ing or `free`ing. Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Can the GC be configured to warn or panic if something is GCed without free()? Then you could detect memory leak bugs by recompiling with Fil-C - with less overhead than valgrind, although I’m guessing still…
Yeah, it panics when you use after free. [1] But I'm not sure it's worth porting your code to Fil-C just to get that property. Because Fil-C still needs to track the memory allocation with its garbage collector. there isn't much advantage to even calling free. If you don't have a use-after-free bug, then it's just adding the overhead of marking the allocation as freed. And if you do have a use-after-free bug, you mig…
> Yeah, it panics when you use after free.
I wasn’t talking about use-after-free, I was talking about memory leaks - when you get a pointer from malloc(), and then you destroy your last copy of the pointer without ever having called free() on it.
Can the GC be configured to warn/panic if it deallocates a memory block which the program failed to explicitly deallocate?
Re: Notes by djb on using Fil-C
#185Earlier quoted context omitted.
It probably uses OS sandboxing primitives already.
In normal operation, apt has to be able to upgrade the kernel, the bootloader, and libc, so it can't usefully be sandboxed except for testing or chroots.
No, that doesn't follow. That only means the networking and parsing functions can't be sandboxed in the same process that drops new root-owned files. C and C++ services have been using subprocesses for sandboxing risky functionality for a long time now. It appears Apt has some version of this:
https://salsa.debian.org/apt-team/apt/-/blob/main/apt-pkg/co...
Re: Notes by djb on using Fil-C
#186Earlier quoted context omitted.
From the topic starter: "I've posted a graph showing nearly 9000 microbenchmarks of Fil-C vs. clang on cryptographic software (each run pinned to 1 core on the same Zen 4). Typically code compiled with Fil-C takes between 1x and 4x as many cycles as the same code compiled with clang" Thus, Fil-C compiled code is 1 to 4 times as slow as plain C. This is not in the "significantly slower" ballpark, like where most inter…
Along with the sibling comment, microbenchmarks should not be used as authoritative data when the use case is full applications. For that matter, highly optimized Java or Go may be "1 to 4 times as slow as plain C". Fil-C has its merits, but they should be described carefully, just with any technology.
Or JavaScript for that matter
Re: Notes by djb on using Fil-C
#187Earlier quoted context omitted.
In normal operation, apt has to be able to upgrade the kernel, the bootloader, and libc, so it can't usefully be sandboxed except for testing or chroots.
No, that doesn't follow. That only means the networking and parsing functions can't be sandboxed in the same process that drops new root-owned files. C and C++ services have been using subprocesses for sandboxing risky functionality for a long time now. It appears Apt has some version of this: https://salsa.debian.org/apt-team/apt/-/blob/main/apt-pkg/co...
That's true; you can't usefully sandbox apt as a whole, but, because it verifies the signatures of the packages it downloads, you could usefully sandbox the downloading process, and you could avoid doing any parsing on the package file until you've validated its signature. It's a pleasant surprise to hear that it already does something like this!
Re: Notes by djb on using Fil-C
#188djb uses a surprisingly low amount of RAM (12GB) considering my laptop already has 64G which is possible to expand to 128G in the future
Re: Notes by djb on using Fil-C
#189Earlier quoted context omitted.
> Like Java, you can just `new` or `malloc` without `delete`ing or `free`ing. Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Can the GC be configured to warn or panic if something is GCed without free()? Then you could detect memory leak bugs by recompiling with Fil-C - with less overhead than valgrind, although I’m guessing still…
Yeah, it panics when you use after free. [1] But I'm not sure it's worth porting your code to Fil-C just to get that property. Because Fil-C still needs to track the memory allocation with its garbage collector. there isn't much advantage to even calling free. If you don't have a use-after-free bug, then it's just adding the overhead of marking the allocation as freed. And if you do have a use-after-free bug, you mig…
Question , would this be a desirable outcome for drivers. As far as i can tell most kernel driver crashes are the ones that would benefit from such protection.Plus obviate the need to do full rewrites - if such a GC can protect from the faults and help with the recovery.Assuming the GC after recovery process is similar to erlang BEAM where a reload can bring back healthy state.
Re: Notes by djb on using Fil-C
#190Does Fil-C catch uninitialized memory reads?