Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes by djb on using Fil-C

cr.yp.to

Post reply on HN
Print view View original

211–220 of 263 posts

Re: Notes by djb on using Fil-C

#211

Post by jitl »

Klonoar wrote: post #112
quotemstr wrote: post #65

I can't wait for all the delicious four-way flamewars. Choose your fighter! 1) Rewrite X in Rust 2) Recompile X using Fil-C 3) Recompile X for WASM 4) Safety is for babies There are a lot of half baked Rust rewrites whose existence was justified on safety grounds and whose rationale is threatened now that HN has heard of Fil-C

Fil-C has come up on HN plenty of times before. If it was going to make much of a dent in the discussions, it would have by now.

odd fallacy. things grow in popularity / awareness over time

Re: Notes by djb on using Fil-C

#212

Post by jitl »

arthur2e5 wrote: post #41
oddmiral wrote: post #26

Earlier quoted context omitted.

I wish, we will have something like Fil-C as an option for unsafe Rust.

Fil-C works because you recompile the whole C userspace. Unsafe Rust doesn't do that... and for many practical purposes you probably want to touch the non-safe-version of the C userspace. Still, it's all LLVM, so perhaps unsafe Rust for Fil-space can be a thing, a useful one for catching (what would be) UBs even [Fil-C defines everything, so no UBs, but I'm assuming you want to eventually run it outside of Fil-space]…

at the bottom of the turtle stack, there's a yolo-c libc that does some syscall stuff:

> libyoloc.so. This is a mostly unmodified [musl/glibc] libc, compiled with Yolo-C. The only changes are to expose some libc internal functionality that is useful for implementing libpizlo.so. Note that libpizlo.so only relies on this library for system calls and a few low level functions. In the future, it's possible that the Fil-C runtime would not have a libc in Yolo Land, but instead libpizlo.so would make syscalls directly.

but mostly you are using a fil-c compiled libc:

> libc.so. This is a modified musl libc compiled with Fil-C. Most of the modifications are about replacing inline assembly for system calls with calls to libpizlo.so's syscall API.

That links here: https://github.com/pizlonator/fil-c/blob/deluge/filc/include...

Quotes from: https://fil-c.org/runtime

Re: Notes by djb on using Fil-C

#213

Post by vacuity »

int_19h wrote: post #201
quotemstr wrote: post #65

I can't wait for all the delicious four-way flamewars. Choose your fighter! 1) Rewrite X in Rust 2) Recompile X using Fil-C 3) Recompile X for WASM 4) Safety is for babies There are a lot of half baked Rust rewrites whose existence was justified on safety grounds and whose rationale is threatened now that HN has heard of Fil-C

It's not an either-or (well, except for this last item). It seems sensible to not write new software in plain C. Rust is certainly a valid choice for a safer language, but in many cases overkill wrt how painful the rewrite is vs benefits gained from avoiding a higher-level memory-safe one like OCaml. At the same time, "let's just rewrite everything!" is also madness. We have many battle-tested libraries written in C…

> the software may be bug-free, but sandboxing it may still be desirable e.g. as a matter of trust (or lack thereof)

Wouldn't the only cause of mistrust be bugs, or am I missing something? If the program is malicious, sandboxing isn't the pertinent action.

Re: Notes by djb on using Fil-C

#214

Post by vacuity »

Findecanor wrote: post #178
kragen wrote: post #24

Earlier quoted context omitted.

So far we haven't found a viable alternative; CHERI has holes in its temporal integrity guarantees.

Both Fil-C and CHERI rely on a concurrent GC/a GC-like task to find and invalidate all pointers to free()'d memory objects (in "quarantine") before putting them back into the memory pool. The difference is that because Fil-C has bounds in each object's header, it only has to nullify it to remove access whereas in CHERI a quarantined object can still be accessed through any pointer that hasn't been invalidated yet. I'…

Implementations can always get better, but I think the only categorical (if even that) improvement that will arise depends on careful program design/implementation; that is, reducing the scope and number of capabilities and providing semantic information on capability usage. Fil-C and CHERI do an admirable job of maximizing backwards compatibility and even allowing incremental improvements, but I think it's time that programmers bought into capabilities too.

Re: Notes by djb on using Fil-C

#215

Post by jitl »

kragen wrote: post #204
jitl wrote: post #202

Earlier quoted context omitted.

the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-... (sqlite is quite a bit smaller than DuckDB tho)

Is it? It's much less new.

for one, duckdb includes all of sqlite (and many other dependencies). it knows how to do things like efficiently query over parquet files in s3. it's expansive - a swiss army knife for working with data wherever it's at.

sqlite is a "self contained system" depending on no external software except c standard library for target os:

> A minimal build of SQLite requires just these routines from the standard C library:

> memcmp(), memcpy(), memmove(), memset(), strcmp(), strlen(), strncmp()

> Most builds also use the system memory allocation routines:

> malloc(), realloc(), free()

> Default builds of SQLite contain appropriate VFS objects for talking to the underlying operating system, and those VFS objects will contain operating system calls such as open(), read(), write(), fsync(), and so forth

Quoting from the appropriately named https://sqlite.org/selfcontained.html

as a very rough and unfair estimate between the two project's source, sqlite is about 8% the size of duckdb:

    $ pwd
    /Users/jitl/src/duckdb/src
    $ sloc .
    
    ---------- Result ------------
    
                Physical :  418092
                  Source :  317274
                 Comment :  50113
     Single-line comment :  46187
           Block comment :  3926
                   Mixed :  4415
     Empty block comment :  588
                   Empty :  55708
                   To Do :  136
    
    Number of files read :  2611
    
    ----------------------------
    $ cd ~/Downloads/sqlite-amalgamation-3500400/
    $ sloc .
    
    ---------- Result ------------
    
                Physical :  34742
                  Source :  25801
                 Comment :  8110
     Single-line comment :  1
           Block comment :  8109
                   Mixed :  1257
     Empty block comment :  1
                   Empty :  2089
                   To Do :  5
    
    Number of files read :  2
    
    ----------------------------

Re: Notes by djb on using Fil-C

#216

Post by pizlonator »

kragen wrote: post #207
pizlonator wrote: post #192

Earlier quoted context omitted.

> Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Wow great question! My intention is to give folks powerful options. You can choose: - Compile your code with Fil-C while still maintaining it for Yolo-C. In that case, you'll be calling free(). Fil-C's free() behavior ensures no GC-induced leaks (more on that below) so code that doe…

Okay, that's brilliant. I didn't even imagine that the GC-induced leak problem was even solvable . I guess the freed-but-not-GCed object could be arbitrarily large, but that's almost never going to be a gradual leak. What's awesome about the Emacs GC?

Even if you have a large freed object, it’ll really get GC’d unless it’s referenced from a root that the GC can’t edit. I allow for such things to simplify the runtime, but they’re rare.

As a GC dev I just found the emacs GC to be so nicely engineered:

- The code is a pleasure to read. I understood it very quickly.

- lots of features! Very sophisticated weak maps, weak references, and finalizers. Not to mention support for heap images (the portable dumper).

- the right amount of tuning but nothing egregious.

It’s super fun to read high quality engineering in an area that I am passionate about!

Re: Notes by djb on using Fil-C

#217

Post by kragen »

jitl wrote: post #215
kragen wrote: post #204

Earlier quoted context omitted.

Is it? It's much less new.

for one, duckdb includes all of sqlite (and many other dependencies). it knows how to do things like efficiently query over parquet files in s3. it's expansive - a swiss army knife for working with data wherever it's at. sqlite is a "self contained system" depending on no external software except c standard library for target os: > A minimal build of SQLite requires just these routines from the standard C library: >…

Oh, wow! I really had no idea!

Re: Notes by djb on using Fil-C

#219

Post by drnick1 »

dontlaugh wrote: post #28
jeffrallen wrote: post #8

Wish we were talking about making Fil-C required for apt, not Rust...

Fil-C is slow. There is no C or C++ memory safe compiler with acceptable performance for kernels, rendering, games, etc. For that you need Rust. The future includes Fil-C for legacy code that isn’t performance sensitive and Rust for new code that is.

No, Rust is awful for game development. It's not really what it was intended for. For one, all the graphics API are in C, so you would have to use unsafe FFI basically everywhere.

Re: Notes by djb on using Fil-C

#220

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #153
1vuio0pswjnm7 wrote: post #88

For those who might miss it, the notes cite a new 64-bit version of cdb that supports exabyte databases https://cdb.cr.yp.to Also maybe of interest is that the new cdb subdomain is using pqconnect instead of dnscurve

RFC 1034 Domain Concepts and Facilities November 1987 [Page 8] "A domain is identified by a domain name, and consists of that part of the domain name space that is at or below the domain name which specifies the domain. A domain is a subdomain of another domain if it is contained within that domain. This relationship can be tested by seeing if the subdomain's name ends with the containing domain's name. For example,…

Correction: s/a CNAME/an alias/
Post reply on HN