Live data from Hacker News

Gleam OTP – Fault Tolerant Multicore Programs with Actors

github.com

71–80 of 89 posts

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#71
post #69
post #56

Earlier quoted context omitted.

I suppose I was unclear. It is OTP-style `gen_server` processes that I'm talking about. > OTP processes communicate via the actor model by sending messages of any type. Each actor is responsible for pattern-matching the incoming message and handling it (or not) based on its type. To implement static typing, you need to know at compile time what type of message an actor can receive, what type it will send back, and ho…

I read the code but I'm not sure I understood all of it (I'm familiar with Elixir, not with Gleam). For normal matters I do believe that your approach works but (start returns the pid of the server, right?) what is it going to happen if something, probably a module written in Elixir or Erlang that wants to prove a point, sends a message of an unsupported type to that pid? I don't think the compiler can prevent that.…

> [...] start returns the pid of the server, right?

Yes, `start` is the part you would stick in a supervision tree, essentially. We start the server so that it can be reached later with the interface functions.

> [...] probably a module written in Elixir or Erlang that wants to prove a point, sends a message of an unsupported type to that pid? I don't think the compiler can prevent that. It's going to crash at runtime or have to handle the unmatched type and return a not implemented sort of error.

Yes, this is already the default behavior of a `gen_server` and is fine, IMO. As a general guideline I would advise against trying to fix errors caused by type-unsafe languages; there is no productive (i.e. long-term fruitful) way to fix a fundamentally unsafe interface (Erlang/Elixir code), the best recourse you have is to write as much code you can in the safe one instead.

Erlang, in Gleam code, is essentially a layer where you put the code that does the fundamentals and then you use the foreign function interface (FFI) to tell Gleam that those functions can be called with so and so types, and it does the type checking. This means that once you travel into Erlang code all bets are off. It's really no different to saying that a certain C function can call assembly code.

    pub type ProcessReference(message, term) {
      ByPid(Pid(message))
      ByGlobal(term)
      ByLocal(Atom)
    }
    
    @external(erlang, "otp_server", "call")
    fn gen_server_call(
      pid: ProcessReference(message, term),
      call_func: CallFunc(state, reply),
    ) -> Result(reply, Nil)
And the corresponding Erlang code:

    call({by_pid, Pid}, CallFunc) ->
        {ok, gen_server:call(Pid, {call_by_func, CallFunc})};
    call({by_global, Name}, CallFunc) ->
        {ok, gen_server:call({global, Name}, {call_by_func, CallFunc})};
    call({by_local, Name}, CallFunc) ->
        {ok, gen_server:call(Name, {call_by_func, CallFunc})}.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#72

For all you wonderful people using Gleam, do you use Phoenix? Or any other web frameworks? I really would love to do Gleam with something like Phoenix, but I have not had enough time to check out all the options yet. I'm in the process of building a new project with Python, but if there's options with Gleam worth looking into I just might try it out for this new project.

There's a talk from Lustre's developer about doing LiveView-type things with Gleam/Lustre. The promise is that you can easily pick and choose which parts of your app run on the client, and which run on the server.

https://www.youtube.com/watch?v=TbCm-zR7qZ0

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#73

gleam would be taken more seriously if they didnt plaster their strong political opinions on the project page..i do not understand why they need to talk about politics on a tech project page..this is not about agreeing or disagreeing with the viewpoints..just keep it out of your professional discourse if you want people to take you or your work seriously...otherwise it just deviates the discourse unnecessarily

> otherwise it just deviates the discourse unnecessarily

You could have chosen not to deviate the discourse.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#74
post #26
post #2

I just started a small project using gleam / lustre, and so far I’m loving it. Worth trying if you’re on the fence, especially if you’re into static types, no nulls, functional, ML type languages. Plus beam of course.

For someone who hasn’t worked with either, is it better to learn gleam/lustre better or elixir/phoenix?

Gleam is cool but honestly for now the ecosystem is so much bigger in Elixir. And yes, you can use some libraries across and things like that, but then again you could also bring in the parts that you need from Gleam into Elixir and instead of vice versa. If you just want to learn a really cool language, I think Gleam is pretty cool. But if you want to learn a language that is more productive but still kind of cool, I would start really like here and then dip my toes into Gleam.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#75

IMHO the actor model is great until you need to share something across processes, then you have a distributed computing problem inside your program. For developing fault tolerant multicore programs I think I'm better off using a functional effects system with software transactional memory like Scala/ZIO than Gleam/OTP. I can still use the actor model where appropriate. Plus the JVM software ecosystem and runtime obse…

Can you give an example where you MUST share mutable data that cannot be passed as immutable data structures? The only thing I can think about is hardcore number crunching and yes that should not be written directly in elixir or Erlang or something like that but you could always write NIFs for that.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#76
post #70

Earlier quoted context omitted.

I'm leading a team that is doing an incremental migration to Gleam by delegating specific parts - we're basically pushing the "functional core, imperative shell" pattern to its macro extreme and having Gleam pick up background jobs from our existing Ruby on Rails codebases where we have heavy calculation tasks. This approach is notably not making proper or really any use of OTP features, but it was an extremely easy…

Did I understand correctly you’re using it for file processing? If so does it yield reliability benefits? We have an assortment of jobs written in Go that process files of various types (CSV, Parquet, TXT) in S3 too. The issue we have is that our Kubernetes jobs crash all the time when they encounter something unexpected. Obviously we should invest into making them more robust but what we really want is some way for…

In our case, the files being processed are datasets that have already been normalized through another ETL tool. Since we're doing the preprocessing ourselves elsewhere, our Gleam parsers are set up to expect a pretty rigid set of inputs. We do all of the file IO / streaming in Elixir and pass the raw data into Gleam as Elixir maps: so Gleam just takes maps, parses them into types pretty rigidly, and our entire Gleam module ecosystem assumes "perfect enough" data.

If we encounter row-level errors in a batch, we log those alongside the outputs. There's nothing particularly intrinsic about out usage of Gleam that prevents the workers from crashing during processing, its all about having error handling set up within the job itself to avoid killing the process or pod running it.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#77

gleam would be taken more seriously if they didnt plaster their strong political opinions on the project page..i do not understand why they need to talk about politics on a tech project page..this is not about agreeing or disagreeing with the viewpoints..just keep it out of your professional discourse if you want people to take you or your work seriously...otherwise it just deviates the discourse unnecessarily

"Black lives matter. Trans rights are human rights. No nazi bullsh*t."

Plaster? Are you referring to this singular line of text way down on their main page? If this statement precludes you from using the project I would imagine it is working as intended.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#78
I learned some Gleam a while ago, my familarity with F# helped, but I didn't find a use case for it. Some while ago I wanted to play with raw sockets so I figured, why not Gleam, its binary parsing syntax will be handy so I looked into the erlang sockets module and attempted to write some ffi wrappers for the necessary functions. Turns out it is not as straight forward, especially that the definitions on erlang side are often complex, with many variants and the dynamic nature doesn't often map very well to Gleam. The documentation was also quite sparse for interop. Interop also involves a lot of Dynamic types so encoding/decoding back which is cumbersome. Sooner or later I hit a wall. By virtue of reading Erlang docs to write the ffi wrappers I eventually got familiar with Erlang syntax which at first was quite alien, but turns out it is rather simple with just some rules to remember. I settled on writing Erlang directly. I am not sure how Elixir compares, never tried it, but I was initially put off by the syntax resembling Ruby, which I have fond memories of, but was always very implicit which I didn't like. Writing pure gleam is pretty fun, but a lot of interesting things require either ffi interop or decoders/encoders even for simple things like json which takes away some of that fun.

Re: Gleam OTP – Fault Tolerant Multicore Programs with Actors

#79
post #78

I learned some Gleam a while ago, my familarity with F# helped, but I didn't find a use case for it. Some while ago I wanted to play with raw sockets so I figured, why not Gleam, its binary parsing syntax will be handy so I looked into the erlang sockets module and attempted to write some ffi wrappers for the necessary functions. Turns out it is not as straight forward, especially that the definitions on erlang side…

If you're enjoying erlang, I highly recommend giving elixir a spin. Maybe take a look at a Phoenix tutorial. Such a joy of a lang to work in. And NIFs in erlang and elixir are so nice re: ffi. OTP is just a great platform.
Post reply on HN