Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

dashbit.co

Post reply on HN
Print view View original

21–30 of 68 posts

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#21

Post by behnamoh »

My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on.

Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well.

So far, Elixir has been even more challenging to just get up and running than Common Lisp!

By the way, the official Elixir website recommends using Homebrew to install it. But almost everyone in the Github issues and comments says ASDF is the way to go.

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#22

Post by cess11 »

davidw wrote: post #3

From a "marketing strategy" point of view, I wonder what the thinking is in investing in this stuff so heavily when Python seems to be kind of the go-to? Will they be able to create a "good enough" environment to do that kind of work with Elixir? Is it just someone or a company scratching their own itch? This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of prog…

It's very serious. The BEAM had a problem in that it lacked solid number-crunching libraries, so some folks solved that, and when they did distributed ML-capabilities kind of just fell out as a neat bonus so some folks did that too.

So now it's integrated into the basic Livebook, just boot it and go to that example and you have a transcriber or whatever as boilerplate to play around with. Want something else from Huggingface? Just switch out a couple of strings referencing that model and all the rest is sorted for you, except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.

Python is awful for sharding, concurrency, distribution, that kind of thing. With the BEAM you can trivially cluster with an instance on a dedicated GPU-outfitted machine that runs the LLM models or what have you and there you have named processes that control process pools for running queries and they'll be immediately available to any BEAM that clusters with it. Fine, you'll do some VPN or something that requires a bit of experience with networking, but compared to building a robust, distributed system in Python it's easy mode.

I don't know what the goals are, but I perceive the Nx/Bumblebee/BEAM platform as obviously better than Python for building production systems. There might be advantages to Python when creating and training models, I'm not sure, but if you already have the models and need to serve more than one, and want the latency to be low so the characteristically slow response feels a little faster, and don't already have a big Kubernetes system for running many Python applications in a distributed manner, then this is for you and it'll be better than good enough until you've created a rather large success.

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#23

Post by cess11 »

behnamoh wrote: post #21

My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on. Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well. So far, Elixir has been even m…

ASDF is easy to use once you've learned a few of the basic commands, and that way you'll have an easy time when new versions come out and want to check out new features. Like the built-in JSON parser, the gradual typing when it drops, things like that.

If you built something useful you might not want to upgrade it just to look at the new stuff, and if you built nothing and just drop into iex a couple of times per year it'll still be easier to pull in the latest BEAM and Elixir versions and play around than figuring out if Homebrew has the new version, and if so, whether it installs nicely over the old or not.

I don't think I've got the LSP running, might check tomorrow. It's OK, the IDE autocompletes some things and for me development basically happens in the REPL and then gets pasted into tests and the project anyway. And iex has good autocomplete and help functions and so on.

Edit: A hurdle might be to install the GUI libraries needed for Observer, but you'll probably be able to search out an incantation for your operating system once you're into doing stuff with process trees.

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#24

Post by clessg »

behnamoh wrote: post #21

My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on. Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well. So far, Elixir has been even m…

Yeah, the LSP situation remains a sore point, which is deeply unfortunate. One of the big reasons I like Gleam! Luckily, there are new contenders popping up to hopefully solve the issues with elixir-ls: try https://github.com/elixir-tools/next-ls or https://github.com/lexical-lsp/lexical. They might give a better experience. In fact, the creator of Elixir recently started directly sponsoring next-ls, so it's probably a safe enough bet.

> By the way, the official Elixir website recommends using Homebrew to install it. But almost everyone in the Github issues and comments says ASDF is the way to go.

The Elixir website is right. Just use Homebrew until you find a real need for asdf or similar tools. It's far simpler.

asdf (or mise[0]) is merely a way to manage different runtime versions between various projects, you would use it the same way as one might use rbenv/rvm, nvm/n, pyenv, or even Docker/nix, and so on. You don't need it until you have several ongoing projects requiring different runtime versions. If you reach that point, great! It'll be worth the effort then, and it isn't difficult.

Personally, I just use Homebrew Elixir for easy ad-hoc access to iex/livebook. If I truly need reproducible environments, devbox[1] (a sort of nix wrapper) is nice and extremely straightforward.

Tl;dr: Just use Homebrew. If your requirements expand beyond that, you probably have far more challenging problems than installing asdf (or whatever).

[0] https://mise.jdx.dev/dev-tools/comparison-to-asdf.html

[1] https://www.jetify.com/devbox

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#25

Post by 6gvONxR4sf7o »

mrdoops wrote: post #17

IMO the big win for Elixir/Nx/Bumblebee/etc is that you can do batched distributed inference out of the box without deploying anything separate to your app or hitting an API. Massive complexity reduction and you can more easily scale up or down. https://hexdocs.pm/nx/Nx.Serving.html#content And there's also a scale to 0 story for when you're not using that GPU at all: https://github.com/phoenixframework/flame 1 langu…

I've been really curious about BEAM languages but never made the leap. How well does it manage heterogeneous compute? I'm used to other languages making me define what happens on CPU vs GPU and defining cross-machine talk around those kinds of considerations.

What parts of that does elixir (and company) allow me to not write? Is there a good balance between abstractions when it comes to still maybe wanting control over what goes where (heteregeneity)?

Super curious and kinda looking for an excuse here :)

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#26

Post by ricketycricket »

cess11 wrote: post #22
davidw wrote: post #3

From a "marketing strategy" point of view, I wonder what the thinking is in investing in this stuff so heavily when Python seems to be kind of the go-to? Will they be able to create a "good enough" environment to do that kind of work with Elixir? Is it just someone or a company scratching their own itch? This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of prog…

It's very serious. The BEAM had a problem in that it lacked solid number-crunching libraries, so some folks solved that, and when they did distributed ML-capabilities kind of just fell out as a neat bonus so some folks did that too. So now it's integrated into the basic Livebook, just boot it and go to that example and you have a transcriber or whatever as boilerplate to play around with. Want something else from Hug…

> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.

May I ask to which service you are referring?

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#27

Post by behnamoh »

clessg wrote: post #24
behnamoh wrote: post #21

My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on. Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well. So far, Elixir has been even m…

Yeah, the LSP situation remains a sore point, which is deeply unfortunate. One of the big reasons I like Gleam! Luckily, there are new contenders popping up to hopefully solve the issues with elixir-ls: try https://github.com/elixir-tools/next-ls or https://github.com/lexical-lsp/lexical . They might give a better experience. In fact, the creator of Elixir recently started directly sponsoring next-ls, so it's probabl…

Thanks! I just checked out devbox, it's so great to have a more user-friendly nix for people like me who don't want to mess with nix!

Also, great point about Gleam! What I like about it, other than types and JS compilation, is exactly the tooling! The devs clearly understood that tooling is extremely important these days and have focused on that. If it's truly inter-operable with Elixir/Erlang libs, I'll probably just use Gleam instead.

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#28

Post by ricketycricket »

6gvONxR4sf7o wrote: post #25
mrdoops wrote: post #17

IMO the big win for Elixir/Nx/Bumblebee/etc is that you can do batched distributed inference out of the box without deploying anything separate to your app or hitting an API. Massive complexity reduction and you can more easily scale up or down. https://hexdocs.pm/nx/Nx.Serving.html#content And there's also a scale to 0 story for when you're not using that GPU at all: https://github.com/phoenixframework/flame 1 langu…

I've been really curious about BEAM languages but never made the leap. How well does it manage heterogeneous compute? I'm used to other languages making me define what happens on CPU vs GPU and defining cross-machine talk around those kinds of considerations. What parts of that does elixir (and company) allow me to not write? Is there a good balance between abstractions when it comes to still maybe wanting control ov…

May not answer all your questions, but this may be a good starting point: https://hexdocs.pm/nx/Nx.Defn.html

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#29

Post by TJSomething »

davidw wrote: post #3

From a "marketing strategy" point of view, I wonder what the thinking is in investing in this stuff so heavily when Python seems to be kind of the go-to? Will they be able to create a "good enough" environment to do that kind of work with Elixir? Is it just someone or a company scratching their own itch? This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of prog…

I feel like a big audience would be people moving away from Spark.

Re: Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.

#30

Post by behnamoh »

cess11 wrote: post #23
behnamoh wrote: post #21

My experience with Elixir onboarding was meh. Spent hours trying to setup the LSP in VSCode and Neovim. Their pseudo-official LSP (elixir-ls) didn't work at all. I even made a post about it on Reddit, Github, and here. No one really knew what was going on. Even with Haskell you have something like ghcup and you're good to go. Not to mention Rust's amazing Cargo and Go's tooling as well. So far, Elixir has been even m…

ASDF is easy to use once you've learned a few of the basic commands, and that way you'll have an easy time when new versions come out and want to check out new features. Like the built-in JSON parser, the gradual typing when it drops, things like that. If you built something useful you might not want to upgrade it just to look at the new stuff, and if you built nothing and just drop into iex a couple of times per yea…

I think the Python dev style I adopted can't be easily ported to Elixir. In Python, I rely heavily on an LSP because I want to fiddle with a lot of functions/classes located deeply in libraries. In VSCode, I simply press CMD and click on any function (or in Neovim, I `gf` or `gd` it). I thought it'd make even more sense in Elixir because apparently everything is a module. Am I missing something? How do you use iex efficiently?
Post reply on HN