Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From Python to Elixir Machine Learning

thestackcanary.com

Post reply on HN
Print view View original

61–70 of 89 posts

Re: From Python to Elixir Machine Learning

#61

Post by throwawaymaths »

mrdoops wrote: post #56
throwawaymaths wrote: post #49

lots of negativity in here from people who have invested a lot of effort learning python, despite that there's also people who complain a lot about ML/DS in python. Is it an expression of sunk cost fallacy? If you don't support people who are trying to make a difference, nothing will change.

It's a weird mental block too - learning a language like Elixir, being so well designed and documented, is as comfortable as it gets. Devs we hire without direct Elixir experience pick it up really quick (within a couple weeks). The energy needed to "get good" with Elixir is really not much considering it provides veritable super powers on the backend and introduces a whole category of concurrency concepts that are n…

specifically over python, immutability in function passing is amazing for beginners and dealing with beginners.

How confident are you that the junior you just hired is operating correctly in the other_code module they're responsible for

    import other_code, as: other

    def f:
        my_dict = {"foo": 1, "bar": 2}
        other.function(my_dict)
        return my_dict["foo"] # ==>  you might be wrong about what's in my_dict

Re: From Python to Elixir Machine Learning

#62

Post by suryong »

gv83 wrote: post #53
pythonaut_16 wrote: post #38

Earlier quoted context omitted.

Could you expand on what you mean by this? Module design doesn't seem any harder than class design in JS or Python. Do you mean the language is generally harder for non-developers? Or that Elixir is harder for JS/Python developers to pick up and write good code? Or something else? Writing well designed Elixir code does seem to require a fairly different approach from most common OO languages, at least at a surface le…

sure thing! I'll try to make points related to what I observed and experienced in my org, starting with one huge huge huge preamble which is: I work in an average org with average joe devs. an average joe dev to me is someone who "just wants a job" and is not very interested in furthering its own professional development or learning new things unless trained & forced by the job. it's perfectly fine to be an average d…

The org I work at has the same issue with Elm, there is one Elm product and no one really wants to take those tickets because everyone hates dealing with it. It has become a total pain in the ass, and I believe someone is rewriting the app in React

Re: From Python to Elixir Machine Learning

#63

Post by lawn »

pawelduda wrote: post #52
pawelduda wrote: post #18

I use Elixir for a long time and wouldn't recommend it for ML over Python. Yes, maybe Elixir has some advantages and solutions for pain points because of VM architecture and other subjective reasons but doesn't come close to what Python offers in terms of tooling, support, community. Also, if you juggle more complex data in Elixir, it's something that you need to get used to coming from Python, it will be much much d…

I can no longer edit parent post so to further elaborate on tooling: even if Elixir was comparable to Python (let's assume it's true as of today), there is a lot going on in AI, so that won't necessarily be true tomorrow. There is no doubt Elixir will always be slower to catch up to recent development because of smaller community. This also applies to how quickly issues will be resolved as they appear. It's likely OK…

If that reasoning would hold true then there would never be a new best/most popular language for a specific domain, but there are many counter examples.

And just as Elixir is (in my opinion) preferable to Python for web development, it's possible that the same may happen with AI.

Re: From Python to Elixir Machine Learning

#64

Post by hosh »

davidktr wrote: post #15
not-my-account wrote: post #10

Earlier quoted context omitted.

You end up having to do a lot of things in a ML training run, some of which you can do in parallel because it’s not important now (eg saving metadata) or because you’d otherwise be resource limited (eg loading data and formatting batches for training)

And for this you cannot use Python's multiprocessing because ... ? Sure, moving data between processes is slow because of pickling [0]. However, I'm using parallel processing for the things you suggested, and for these it works great. If I really had the use case and needed threads, I'd much rather use C++ bindings in a Python package than rebuilding the whole thing. Guess it depends on the scale we are talking about…

It’s handling all the things that can go wrong when communicating and coordinating across processes, across machines, or troubleshooting bottlenecks on running systems that Elixir (and Erlang) excels.

Re: From Python to Elixir Machine Learning

#65

Post by parthdesai »

gv83 wrote: post #53
pythonaut_16 wrote: post #38

Earlier quoted context omitted.

Could you expand on what you mean by this? Module design doesn't seem any harder than class design in JS or Python. Do you mean the language is generally harder for non-developers? Or that Elixir is harder for JS/Python developers to pick up and write good code? Or something else? Writing well designed Elixir code does seem to require a fairly different approach from most common OO languages, at least at a surface le…

sure thing! I'll try to make points related to what I observed and experienced in my org, starting with one huge huge huge preamble which is: I work in an average org with average joe devs. an average joe dev to me is someone who "just wants a job" and is not very interested in furthering its own professional development or learning new things unless trained & forced by the job. it's perfectly fine to be an average d…

> It also requires a different way of exploring code as you can't do the familiar `.` and see what happens, you can't just do `price.toCentsValue` but need to do `price |> Price.toCentsValue` and so you need to know the existence of the `Price` module, which might not exist and be buried in `Cart` or as an helper in some controller because you did not understand clearly the domain and the modules responsibilities. Attaching behaviour to data is powerful, explorable, and most people are used to it, even if it's the wrong place in principle, with modules this is flipped, it's now data that must thread through operations, and it's not super easy to grasp.

Not sure, I think you're picking and choosing things to ramble about. While you can't just do `price.toCentsValue`, and have to call `Price` module, in a OO language, you would need to instantiate `price = new Price(amount 10)` or something before calling `toCentsValue`. This means you're aware of the Price class, same as being aware of the Price module. If anything being aware of the module infact is better IMO since it allows you trace through explicitly what the code is going to do. Your point is correct if you're working through macros though.

Re: From Python to Elixir Machine Learning

#66

Post by parthdesai »

suryong wrote: post #62
gv83 wrote: post #53

Earlier quoted context omitted.

sure thing! I'll try to make points related to what I observed and experienced in my org, starting with one huge huge huge preamble which is: I work in an average org with average joe devs. an average joe dev to me is someone who "just wants a job" and is not very interested in furthering its own professional development or learning new things unless trained & forced by the job. it's perfectly fine to be an average d…

The org I work at has the same issue with Elm, there is one Elm product and no one really wants to take those tickets because everyone hates dealing with it. It has become a total pain in the ass, and I believe someone is rewriting the app in React

That's going to be the case with almost any legacy project though

Re: From Python to Elixir Machine Learning

#67

Post by codybontecou »

mrdoops wrote: post #56
throwawaymaths wrote: post #49

lots of negativity in here from people who have invested a lot of effort learning python, despite that there's also people who complain a lot about ML/DS in python. Is it an expression of sunk cost fallacy? If you don't support people who are trying to make a difference, nothing will change.

It's a weird mental block too - learning a language like Elixir, being so well designed and documented, is as comfortable as it gets. Devs we hire without direct Elixir experience pick it up really quick (within a couple weeks). The energy needed to "get good" with Elixir is really not much considering it provides veritable super powers on the backend and introduces a whole category of concurrency concepts that are n…

Do you know which resources your devs are using to learn Elixir from scratch?

Re: From Python to Elixir Machine Learning

#68

Post by throwawaymaths »

catchnear4321 wrote: post #42
pythonaut_16 wrote: post #38

Earlier quoted context omitted.

Could you expand on what you mean by this? Module design doesn't seem any harder than class design in JS or Python. Do you mean the language is generally harder for non-developers? Or that Elixir is harder for JS/Python developers to pick up and write good code? Or something else? Writing well designed Elixir code does seem to require a fairly different approach from most common OO languages, at least at a surface le…

your last paragraph largely answers your questions. it doesn’t take much to yeet some python scripts (and more) at the wall and get something to stick. SO, GPT, pick your poison, it’s painfully easy. solved problems as far as the eye can see, with a little glue or tape to hold it together. elixir demands more of an investment. more than approximately zero is quite a bit if you already have momentum. (elixir can absol…

Honestly long run elixir code by LLMs is going to be better because the llm won't have enough attention or context to be sure that mutable passes to functions don't absolutely wreck your data

Re: From Python to Elixir Machine Learning

#69

Post by throwawaymaths »

gv83 wrote: post #54
lvass wrote: post #51

Earlier quoted context omitted.

On the other hand, it may filter less dedicated people from showing up at your door.

this has been a point of contention with numerous hr departments that needs to hire fast "cuz investors!!11". this is the reality of life sadly

This is the real problem.

Most orgs that fail on elixir fail due to management.

Re: From Python to Elixir Machine Learning

#70

Post by mrdoops »

codybontecou wrote: post #67
mrdoops wrote: post #56

Earlier quoted context omitted.

It's a weird mental block too - learning a language like Elixir, being so well designed and documented, is as comfortable as it gets. Devs we hire without direct Elixir experience pick it up really quick (within a couple weeks). The energy needed to "get good" with Elixir is really not much considering it provides veritable super powers on the backend and introduces a whole category of concurrency concepts that are n…

Do you know which resources your devs are using to learn Elixir from scratch?

* Elixir-lang Intro: https://elixir-lang.org/getting-started/introduction.html

* Exercism track: https://exercism.org/tracks/elixir

* Sasa Juric's book: https://www.manning.com/books/elixir-in-action-third-edition

* Dave Thomas's Elixir Course: https://codestool.coding-gnome.com/courses/elixir-for-progra...

* Phoenix Guides: https://hexdocs.pm/phoenix/overview.html

* Ecto Guides: https://hexdocs.pm/ecto/getting-started.html

The above covers the language basics/ideas/concepts and the main tooling (Phoenix/Ecto) if you're looking to build apps or get an Elixir job. I definitely recommend the Phoenix Guides or similar - they're very high quality and kept up to date with any new releases or changes while books can sometimes get out of date.

Post reply on HN