Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes by djb on using Fil-C

cr.yp.to

Post reply on HN
Print view View original

241–250 of 263 posts

Re: Notes by djb on using Fil-C

#241

Post by vacuity »

int_19h wrote: post #238
vacuity wrote: post #213

Earlier quoted context omitted.

> the software may be bug-free, but sandboxing it may still be desirable e.g. as a matter of trust (or lack thereof) Wouldn't the only cause of mistrust be bugs, or am I missing something? If the program is malicious, sandboxing isn't the pertinent action.

If any program can potentially be malicious (which is the effectively the case today with any downloaded software), then sandboxing is exactly the pertinent action - provided that the sandbox is tight enough.

I should have elaborated. If a program is known to be malicious, or should be treated as malicious, then it should probably be terminated. Given a potentially malicious program and no easy way to determine (lack of) malice, sandboxing is a reasonable measure.

Re: Notes by djb on using Fil-C

#243

Post by cb321 »

galangalalgol wrote: post #157
summarity wrote: post #139

Earlier quoted context omitted.

In terms of GC quality, Nim comes to mind.

I keep ignoring nim for some reason. How fast is it with all the checks on? The benchmarks for it julia, and swift typically turn off safety checks, which is not how I would run them.

Since anything/0 = infinity, these kinds of things always depend upon what programs do and as a sibling comment correctly observes how much they interfere with SIMD autovectorization and sevral other things.

That said, as a rough guideline, nim c -d=release can certainly be almost the same speed as -d=danger and is often within a few (single digits) percent. E.g.:

    .../bu(main)$ nim c -d=useMalloc --panics=on --cc=clang -d=release -o=/t/rel unfold.nim
    Hint: mm: orc; opt: speed; options: -d:release
    61608 lines; 0.976s; 140.723MiB peakmem; proj: .../bu/unfold.nim; out: /t/rel [SuccessX]
    .../bu(main)$ nim c -d=useMalloc --panics=on --cc=clang -d=danger -o=/t/dan unfold.nim
    Hint: mm: orc; opt: speed; options: -d:danger
    61608 lines; 2.705s; 141.629MiB peakmem; proj: .../bu/unfold.nim; out: /t/dan [SuccessX]
    .../bu(main)$ seq 1 100000 > /t/dat
    .../bu(main)$ /t
    /t$ re=(chrt 99 taskset -c 2 env -i HOME=$HOME PATH=$PATH)
    /t$ $re tim "./dan -n50 /n" "./rel -n50 /n"
    225.5 +- 1.2 μs (AlreadySubtracted)Overhead
    4177 +- 15 μs   ./dan -n50 /n
    4302 +- 17 μs   ./rel -n50 /n
    /t$ a (4302 +- 17)/(4177 +- 15)
    1.0299 +- 0.0055
    /t$ a 299./55
    5.43636... # kurtosis=>5.4 sigmas is not so significant
Of course, as per my first sentence, the best benchmarks are your own applications run against your own data and its idiosyncratic distributions.

EDIT: btw, /t -> /tmp which is a /dev/shm bind mount while /n -> /dev/null.

Re: Notes by djb on using Fil-C

#244

Post by gkfasdfasdf »

pizlonator wrote: post #192
skissane wrote: post #174

Earlier quoted context omitted.

> Like Java, you can just `new` or `malloc` without `delete`ing or `free`ing. Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Can the GC be configured to warn or panic if something is GCed without free()? Then you could detect memory leak bugs by recompiling with Fil-C - with less overhead than valgrind, although I’m guessing still…

> Is your intention that people use the Fil-C garbage collector instead of free()? Or is it just a backstop in case of memory leak bugs? Wow great question! My intention is to give folks powerful options. You can choose: - Compile your code with Fil-C while still maintaining it for Yolo-C. In that case, you'll be calling free(). Fil-C's free() behavior ensures no GC-induced leaks (more on that below) so code that doe…

Is it possible to use Fil-C as a replacement for valgrind/address sanitizer/leak sanitizer? I.e. say I have a C program that does manual memory management already. Can I then compile it with Fil-C and have it panic/assert on heap use after free, uninitialized memory read (including stack), array out of bounds read, etc?

Re: Notes by djb on using Fil-C

#245

Post by cb321 »

lproven wrote: post #234
quotemstr wrote: post #64

Earlier quoted context omitted.

It's amazing how much technical discourse revolves around impressions. "Oh, it has a GC! GC bad!" "No, this GC by smart guy, so good!" "No, GC always bad!" People aren't engaging with the technical substance. GC based systems and can be plenty good and fast. How do people think JavaScript works? And Go? It's like people just absorbed from the discursive background radiation the idea GC is slow without understanding w…

> It's amazing how much technical discourse revolves around impressions. One of the single most incisive comments in the whole discussion. My take: people don't take the time to even try to understand some things of only moderate complexity. They dismiss it as "too hard", drop it, accept the received wisdom and move on. This is also behind the curse of "best practice". After coming up on 40Y in the industry, my impre…

While I agree with everything you wrote, the "better ways" have a "cap" on how effective they can be. The root causes -- both in-group preference[1] and laziness/delegation to the "smart loudmouth contemporaries/smart-enough predecessors" -- will be things for the foreseeable future among humans. Some might even call them eternal / instinctive. Our whole civilization is based upon delegation/layering, but trust sure is tricky! Even the smartest humans fall prey to Gell-Mann amnesia[2] on topics beyond their expertise. Personally, I think most of what you wrote all connects to the cluster of wicked problems[3] that I think of as "Humanity Complete" (after NP-Complete transformability).

[1] https://en.wikipedia.org/wiki/In-group_favoritism

[2] https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

[3] https://en.wikipedia.org/wiki/Wicked_problem

Re: Notes by djb on using Fil-C

#246

Post by skissane »

pizlonator wrote: post #237
skissane wrote: post #224

Earlier quoted context omitted.

> Nope. Reason: the Fil-C runtime itself now relies on GC, and there's some functionality that only a GC can provide that has proven indispensable for porting some complex stuff (like CPython and Perl5). What if there was a flag you could set on an allocation, “must be freed”. An app can set the “must be freed” flag on its allocations, meaning when the GC collects the allocation, it checks if free() has been called o…

Internal allocations can have pointers to user allocations

Right… but how is that a problem for my proposal? I’m not seeing the issue.

Re: Notes by djb on using Fil-C

#247

Post by loeg »

HexDecOctBin wrote: post #223
jitl wrote: post #200

Earlier quoted context omitted.

> There is no interoperability with Yolo-C (i.e. classic C). This is both a goal and the outcome of a non goal. https://fil-c.org/runtime (worth reading, i think all the stuff Fil writes is both super informative & quite entertaining.)

This is disappointing. I can write the networking parts in Rust and the rest of the program in C, but apparently can't do the same with Fil-C.

You could run the two parts in separate processes and pass messages via shared memory.

Re: Notes by djb on using Fil-C

#248

Post by lproven »

cb321 wrote: post #245
lproven wrote: post #234

Earlier quoted context omitted.

> It's amazing how much technical discourse revolves around impressions. One of the single most incisive comments in the whole discussion. My take: people don't take the time to even try to understand some things of only moderate complexity. They dismiss it as "too hard", drop it, accept the received wisdom and move on. This is also behind the curse of "best practice". After coming up on 40Y in the industry, my impre…

While I agree with everything you wrote, the "better ways" have a "cap" on how effective they can be. The root causes -- both in-group preference[1] and laziness/delegation to the "smart loudmouth contemporaries/smart-enough predecessors" -- will be things for the foreseeable future among humans. Some might even call them eternal / instinctive. Our whole civilization is based upon delegation/layering, but trust sure…

Very nicely put. Thanks for that.

I'd offer solutions, except for the trivial implementation detail that I don't have any. But then, if I did, I'd have a Nobel and possibly be the first president of the united planet.

Re: Notes by djb on using Fil-C

#249

Post by silotis »

thesz wrote: post #232
silotis wrote: post #152

Earlier quoted context omitted.

This depends heavily on what problem domain you're talking about. For example, a DBMS is necessarily going to shuffle a lot of data into and out of memory.

I am professional database developer. We do not do what you are thinking we are doing. ;)

I was thinking less about the DB data itself and more about temporary allocations that have to be made per-request. The same is true for most server software. Even if arenas are used to reduce the number of allocations you're still doing a lot more memory management than a typical cryptographic benchmark.

Re: Notes by djb on using Fil-C

#250

Post by zX41ZdbW »

jitl wrote: post #202
kragen wrote: post #179

Earlier quoted context omitted.

> DuckDB is a huge pile of C++. I don't see a RIIR taking off before AGI. While I'm not a big fan of rewriting things, all of DuckDB has been written in the last 10 years. Surely a rewrite with the benefit of hindsight could reach equivalent functionality in less than 10 years?

the sqlite RIIR is going quite well: https://turso.tech/blog/beyond-the-single-writer-limitation-... (sqlite is quite a bit smaller than DuckDB tho)

I'm trying to make Turso load some data, but it is so slow that even several months are not enough to load the dataset: https://github.com/ClickHouse/ClickBench/issues/336
Post reply on HN