Live data from Hacker News

Zigler: Zig NIFs in Elixir

github.com

31–40 of 93 posts

Re: Zigler: Zig NIFs in Elixir

#31
post #28
post #26

Earlier quoted context omitted.

And then there are port drivers which are the worst of both worlds! Can crash the BEAM and need much more ceremony than NIF to set up but they’re pretty nice to do in Zig[1] as well [1]: https://github.com/borgoat/er_zig_driver

That's true. Haha! There's another option and that's setting up an Erlang node in the other language. The Erlang term format is relatively straightforward. But I'm honestly not sure of the benefit of a node versus just using a port.

Node:

- can "easily" send beam terms back and forth

- if you want it to be os-supervised separately (systemd, kubernetes, e.g.)

- pain in the ass

Port:

- easy

- usually the only choice if you're not the software author

- really only communicates via stdio bytestreams

- risk of zombies if... Iirc the stdout is not closed properly?

- kind of crazy how it works, Erlang VM spawns a separate process as a middleman

Re: Zigler: Zig NIFs in Elixir

#32
post #21
post #10

Earlier quoted context omitted.

Hence why Rustler is of so much interest since it provides more protections against this happening. Discord is a big Erlang + Rustler user.

Are they really? Their projects don't look so active

I wrote sorted_set_nif, the lack of activity isn’t a lack of care about the library but more just a reflection that the library is done.

With data structures that have some definite behavior unless someone finds a defect there isn’t going to be much activity.

Re: Zigler: Zig NIFs in Elixir

#33
post #30
post #5

I use zig a lot in elixir nif, for things like audio and video processing, it works great. But I do not use zigler as I prefer the code to live in their own codebases. But zigler is really nice and it provides an easy way to do computational heavy tasks in elixir.

> I use zig a lot in elixir nif, for things like audio and video processing Sounds interesting, is it open source? I am interested in seeing how the code layout looks like when mixing Zig and Elixir

I don't have open source code base to share but here it how it looks like:

        // the_nif.zig

        fn init_imp(
            env: ?*erl.ErlNifEnv,
            argc: c_int,
            argv: [*c]const erl.ERL_NIF_TERM,
        ) !erl.ERL_NIF_TERM {
            if (argc != 0) {
                return error.BadArg;
            }

            return try helpers.make("Hello world");
        }

        export fn media_tools_init(
            env: ?*erl.ErlNifEnv,
            argc: c_int,
            argv: [*c]const erl.ERL_NIF_TERM,
        ) erl.ERL_NIF_TERM {
            return init_imp(env, argc, argv) catch |err|
                return helpers.make_error(env, err);
        }


        var funcs = [_]erl.ErlNifFunc{ erl.ErlNifFunc{
            .name = "init",
            .arity = 1,
            .fptr = media_tools_init,
            .flags = erl.ERL_NIF_DIRTY_JOB_CPU_BOUND, 
        } };

        var entry = erl.ErlNifEntry{
            .major = erl.ERL_NIF_MAJOR_VERSION,
            .minor = erl.ERL_NIF_MINOR_VERSION,
            .name = "Elixir.MediaTools.Stream",
            .num_of_funcs = funcs.len,
            .funcs = &funcs,
            .load = load,
            .reload = null,
            .upgrade = null,
            .unload = null,
            .vm_variant = "beam.vanilla",
            .options = 0,
            .sizeof_ErlNifResourceTypeInit = @sizeOf(erl.ErlNifResourceTypeInit),
            .min_erts = "erts-10.4",
        };

        export fn nif_init() *erl.ErlNifEntry {
            return &entry;
        }

        # the_exlixir_file.ex

        assert "Hello world" == MediaTools.Stream.init()

The "helpers" library is used to convert types to and from erlang, I plan on open sourcing it but it is not ready now. In the above example, the code is explicit but "entry" can be created with an helper comptime function. erl is simply the erl_nif.h header converted by zig translate-c.

I wrote a piece back in 2022, but things evolved a lot since then: https://www.kuon.ch/post/2022-11-26-zig-nif/

Re: Zigler: Zig NIFs in Elixir

#34
post #10
post #9

Earlier quoted context omitted.

It’s important to note that while Erlang has protections against user code crashing an Erlang process and recovering, a faulty NIF can take down the entire virtual machine.

Hence why Rustler is of so much interest since it provides more protections against this happening. Discord is a big Erlang + Rustler user.

What kind of protections as opposed to Zigler?

Re: Zigler: Zig NIFs in Elixir

#35
post #21

Earlier quoted context omitted.

Are they really? Their projects don't look so active

It’s pretty common in the Elixir ecosystem for these types of libraries to not change very much. Elixir itself doesn’t change too much so these libraries stay solid without needing frequent updates. It doesn’t mean people aren’t using them. Some libraries even put disclaimers that they are actively maintained even if they haven’t seen an update in a long time. It’s something that takes some getting used to for some p…

Yep. This is one reason I choose Elixir for a project. For a variety of use cases, long term stability is a big plus.

Re: Zigler: Zig NIFs in Elixir

#36
post #10

Earlier quoted context omitted.

Hence why Rustler is of so much interest since it provides more protections against this happening. Discord is a big Erlang + Rustler user.

What kind of protections as opposed to Zigler?

Rust comes with memory safety.

It's one less potential cause that might bring down the entire Erlang VM.

Re: Zigler: Zig NIFs in Elixir

#37
post #33
post #30

Earlier quoted context omitted.

> I use zig a lot in elixir nif, for things like audio and video processing Sounds interesting, is it open source? I am interested in seeing how the code layout looks like when mixing Zig and Elixir

I don't have open source code base to share but here it how it looks like: // the_nif.zig fn init_imp( env: ?*erl.ErlNifEnv, argc: c_int, argv: [*c]const erl.ERL_NIF_TERM, ) !erl.ERL_NIF_TERM { if (argc != 0) { return error.BadArg; } return try helpers.make("Hello world"); } export fn media_tools_init( env: ?*erl.ErlNifEnv, argc: c_int, argv: [*c]const erl.ERL_NIF_TERM, ) erl.ERL_NIF_TERM { return init_imp(env, argc,…

Thanks for sharing the post, it was intriguing. The detailed comments mentioned in `main.zig` and `build.zig` towards the end helped a lot.

Re: Zigler: Zig NIFs in Elixir

#38
(Yo dawg, we put a niche language into a niche language so that...)

I wonder if the Zig code can be not written inline, as an option. With anything larger than a few lines, I'd want syntax highlighting, LSP support, navigation, etc. It's easier to achieve with one language per file.

Re: Zigler: Zig NIFs in Elixir

#39
post #25

Zig is also used in an excellent way by burrito[0]. I've also used zig for compiling NIFs written in C/C++/Objective-C, since `zig cc` makes cross-compiling much nicer. I wish zig got more use and attention in the Erlang ecosystem, but rustler seems more popular.

[0] https://github.com/burrito-elixir/burrito

Re: Zigler: Zig NIFs in Elixir

#40
post #21

Earlier quoted context omitted.

Are they really? Their projects don't look so active

It’s pretty common in the Elixir ecosystem for these types of libraries to not change very much. Elixir itself doesn’t change too much so these libraries stay solid without needing frequent updates. It doesn’t mean people aren’t using them. Some libraries even put disclaimers that they are actively maintained even if they haven’t seen an update in a long time. It’s something that takes some getting used to for some p…

> It’s pretty common in the Elixir ecosystem for these types of libraries to not change very much.

This is kind of fascinating and seems worthy of more detailed study. I'm sure almost anything looks stable compared to javascript/python ecosystems, but would be interesting to see how other ecosystems with venerable old web-frameworks or solid old compression libraries compare. But on further reflection.. language metrics like "popularity" are also in danger of just quantifying the churn that it takes to keep working stuff working. You can't even measure strictly new projects and hope that helps, because new projects may be a reaction to perceived need to replace other stuff that's annoyingly unstable over periods of 5-10 years, etc.

Some churn is introduced by trying to keep up with a changing language, standard lib, or other dependencies, but some is just adding features forever or endlessly refactoring aesthetics under different management. Makes me wish for a project badge to indicate a commitment like finished-except-for-bugfixes.

Post reply on HN