Earlier quoted context omitted.
I come from Ruby but the reactions can be similar, happy to give my data point. The thing is Elixir is really good at an increasing number of things. If you need to write a HTTP proxy in the middle of your application, since Elixir processes & incoming HTTP workers are cheap, you do not need to go evented: it just works. If you need to have reactive web apps with automated changes pushed to the client, it's the same:…
still not easy at the most important thing of all: being approachable, instrumented and intuitive to people less dedicated to programming. I say this as someone that likes elixir, but after seeing it failing miserably at my org, I'm very skeptical it can be thrown around like a spring or node or django project. It needs real support from the org and requires module design skills that are not present in most random de…
From Python to Elixir Machine Learning
51–60 of 89 posts
Re: From Python to Elixir Machine Learning
#52I use Elixir for a long time and wouldn't recommend it for ML over Python. Yes, maybe Elixir has some advantages and solutions for pain points because of VM architecture and other subjective reasons but doesn't come close to what Python offers in terms of tooling, support, community. Also, if you juggle more complex data in Elixir, it's something that you need to get used to coming from Python, it will be much much d…
I like Elixir for web development otherwise, it is a much more stable domain so above doesn't apply (although I've seen some claim otherwise, which is telling how much more of an issue it would be for niche ML use case).
I'd be very happy to be proven wrong by some case studies of how companies leveraged Elixir in real ML projects and concluded it is superior to Python.
Re: From Python to Elixir Machine Learning
#53Earlier quoted context omitted.
still not easy at the most important thing of all: being approachable, instrumented and intuitive to people less dedicated to programming. I say this as someone that likes elixir, but after seeing it failing miserably at my org, I'm very skeptical it can be thrown around like a spring or node or django project. It needs real support from the org and requires module design skills that are not present in most random de…
Could you expand on what you mean by this? Module design doesn't seem any harder than class design in JS or Python. Do you mean the language is generally harder for non-developers? Or that Elixir is harder for JS/Python developers to pick up and write good code? Or something else? Writing well designed Elixir code does seem to require a fairly different approach from most common OO languages, at least at a surface le…
having said that, designing a proper elixir module (which basically is a bunch of functions that operates on a certain data structure usually represented by map of a certain shape) carries a certain level of fundamental understanding of the operation you're doing, which in my experience is one of the hardest things to get correctly.
It also requires a different way of exploring code as you can't do the familiar `.` and see what happens, you can't just do `price.toCentsValue` but need to do `price |> Price.toCentsValue` and so you need to know the existence of the `Price` module, which might not exist and be buried in `Cart` or as an helper in some controller because you did not understand clearly the domain and the modules responsibilities. Attaching behaviour to data is powerful, explorable, and most people are used to it, even if it's the wrong place in principle, with modules this is flipped, it's now data that must thread through operations, and it's not super easy to grasp.
Also, tooling is not that good (dialyzer sucks, intellij plugin not that good, vscode lsp good but still not a proper experience from people coming from c#/java, type annotations are not that readable...), pattern matching and destructuring on fn arguments confuses people and it's not super easy to read, and a million other papercuts related to tooling and syntax.
We don't have many elixir codebases (let's say around 15-25%) and I've seen incessant whining about "we don't want to maintain elixir" simply because the majority of people cannot be bothered to learn another mountain of quirks and papercuts (every lang has them) plus also losing the familiar way of working they already have, and having to remember that for the spot ticket that appears once in a blue moon on jira for elixir. That's why I think elixir needs extra support from the organization, basically in mandating it to be the primary language, teaching people proper design + proper code navigation and structure techniques, etc.
I hope I've been clear in my long winded ramblings; and I still wish a great future for elixir, so it becomes more approachable in average places like mine
Re: From Python to Elixir Machine Learning
#54Earlier quoted context omitted.
still not easy at the most important thing of all: being approachable, instrumented and intuitive to people less dedicated to programming. I say this as someone that likes elixir, but after seeing it failing miserably at my org, I'm very skeptical it can be thrown around like a spring or node or django project. It needs real support from the org and requires module design skills that are not present in most random de…
On the other hand, it may filter less dedicated people from showing up at your door.
Re: From Python to Elixir Machine Learning
#55Earlier quoted context omitted.
I come from Ruby but the reactions can be similar, happy to give my data point. The thing is Elixir is really good at an increasing number of things. If you need to write a HTTP proxy in the middle of your application, since Elixir processes & incoming HTTP workers are cheap, you do not need to go evented: it just works. If you need to have reactive web apps with automated changes pushed to the client, it's the same:…
And nerves makes it easy to do IoT stuff / almost-embedded
Re: From Python to Elixir Machine Learning
#56lots of negativity in here from people who have invested a lot of effort learning python, despite that there's also people who complain a lot about ML/DS in python. Is it an expression of sunk cost fallacy? If you don't support people who are trying to make a difference, nothing will change.
Devs we hire without direct Elixir experience pick it up really quick (within a couple weeks). The energy needed to "get good" with Elixir is really not much considering it provides veritable super powers on the backend and introduces a whole category of concurrency concepts that are not easy to grasp elsewhere.
Re: From Python to Elixir Machine Learning
#57Instead of moving to Elixir I believe it makes more sense to wait/move to Mojo when it's ready: https://www.modular.com/mojo
Elixir & Python are not an apples to apples comparison - there are fundamental differences in the programming model (functional, immutability, etc) and runtime (preemptive scheduling + OTP) that is the reason it has distinct advantages not available without heavy cost trade-offs elsewhere.
Either way once Mojo is production ready Elixir will be able to use it as well like it does Rust, Zig, or Python.
Re: From Python to Elixir Machine Learning
#58Earlier quoted context omitted.
I found explorer quite frustrating. I've used polars in python and loved it, but I brought in some financial data and couldn't strip off "£" from the start of a string so I could go on to cast it to a number. As far as I could tell I would have to bring that data into elixir, do the text processing and put it back into explorer, which to me defeated the whole point of a dataframe library. I imagine it's good for prec…
José from the Nx/Explorer team here. Feel free to open up an issue! We have been focused more on high-level features (such as integration with S3, Postgres, Snowflake, SQLite, etc) and therefore we are missing many functions that already exist on Polars. Good news is that it is very quick to add them, so just let us know. :)
Yeah I noticed that, it looks like the numeric manipulations are well represented, less so for strings.
I'll dig out the the code later on and get an issue raised.
Re: From Python to Elixir Machine Learning
#59Earlier quoted context omitted.
I found explorer quite frustrating. I've used polars in python and loved it, but I brought in some financial data and couldn't strip off "£" from the start of a string so I could go on to cast it to a number. As far as I could tell I would have to bring that data into elixir, do the text processing and put it back into explorer, which to me defeated the whole point of a dataframe library. I imagine it's good for prec…
You can use str.slice or str.extract to clean the data: https://pola-rs.github.io/polars/py-polars/html/reference/ex...
Re: From Python to Elixir Machine Learning
#60I wish there wouldn’t be such a song and dance about “moving away from Python”. There’s nothing wrong with creating ML tools in Elixir, but it’s always Python is slow, Python has no concurrency support, blah blah