141–150 of 291 posts
Earlier quoted context omitted.
I know what you mean. I especially dislike the use of an "end" keyword everywhere without a corresponding "begin" keyword.
I think that's inherited from matlab.
Earlier quoted context omitted.
This is a big part of it. People who are scientists write code that we would think is disgusting and don’t care that much about abstraction outside of mathematical functions. There’s a lot to learn with Java. I helped classmates in my intro to programming class when I went to college, because I already knew how to code, and I have no idea why they picked that language as an introduction language. After weeks people w…
> People who are scientists write code that we would think is disgusting and don’t care that much about abstraction outside of mathematical functions. - plenty of scientists write good code; I think the "scientist can't code" meme is harmful. - most of the people that write PyTorch code aren't necessarily scientists - they're software developers (data scientists, ML engineers, research engineers, whatever title - but…
What stage of Julia denial is this?
But, then what?
I could not use it anywhere I worked. The ecosystem was lacking.
Julia is good, but for what exactly?
People involved with Julia are always big with words, but when will I see it in use somewhere?
> Julia says: > A language must compile to efficient code, and we will add restrictions to the language (type stability) to make sure this is possible. > A language must allow post facto extensibility (multiple dispatch), and we will organize the ecosystem around JIT compilation to make this possible. > The combination of these two features gives you a system that has dynamic language level flexibility (because you h…
That’s a really good question and I’m not sure I fully understand all the reasons. One big one is that Java intentionally has made interop with native libraries quite difficult. One cannot take a language seriously for numerical computing if you can’t easily call BLAS, LAPACK and FTTW just to start, and there’s a neverending supply of such efficient native libraries. Julia, on the other hand makes it easy to write ef…
This is mostly true, but the primitives are value types and you can get some things done with them. (Not enough to make Java good for these use cases, no.) I.e. write float[] instead of Float[] and you have a contiguously allocated region of memory that can be efficiently accessed.
Earlier quoted context omitted.
Language choices are less about language to me and more about ecosystem of libraries. Python has generally been very strong in the ml/data science realm. I know Julia is catching up but am unsure just how much it covers. Example of python libraries I would consider needed as part of the data ecosystem, numpy, pytorch/tf, batch and stream processing frameworks like spark/flink/beam, workflow orchestration like kubeflo…
One of the really nice things with Julia is some of the ecosystem needs disappear. Python needs a lot of ecosystem because none of the packages work together, and the language is slow so you have to make sure you are doing as much work as possible outside the language itself. To answer your question more specifically: Numpy -> Array + broadcasting (both in Julia Base) pytoch/tf -> Flux.jl (package) batch/stream proce…
What’s workflow orchestration choice? That’s main one you didn’t touch. My work area in on an ml training platform and a lot of my work can be described as wrapper work on kubeflow to allow dozens of other ml engineers to manage experiments/workflows. For python the main choices are kubeflow/airflow. Ray kind of but ray workflows are still quite new and missing a lot of useful features. I need some system to run hundreds of ml workflows (one workflow being like 5-10 tasks some short some long) per day and manage there tasks well.
Broader area also includes libraries like weights and biases, bento ml, etc (experimentation management libraries).
In theory you can have workflow manager in one language and workflow code in a different language. Main downside is it makes debugging locally workflows harder (breakpoints are a little sad across most language boundaries), but it is doable and we debated migrating to temporal (Java workflow system) before.
Earlier quoted context omitted.
One of the really nice things with Julia is some of the ecosystem needs disappear. Python needs a lot of ecosystem because none of the packages work together, and the language is slow so you have to make sure you are doing as much work as possible outside the language itself. To answer your question more specifically: Numpy -> Array + broadcasting (both in Julia Base) pytoch/tf -> Flux.jl (package) batch/stream proce…
Good spark support would be a good answer for batch/stream processing. I am a little scared of definition of support. Apache beam supports like 5 runners (flink, spark, data flow, etc) but the quality of runner support is extremely inconsistent. I’ve also noticed even for python flink sometimes have very useful operations only in Java with no wrapper. Although honestly having data pipelines in one language and downst…
One of the really big potential benefits of Julia is that it lets you remove language barriers which is especially nice if you are doing ML research (or playing around with new model types, etc). Since the ML stack is Julia all the way down to CUDA/BLAS/julia loops, you can really easily inspect or modify everything in your stack.
Earlier quoted context omitted.
For numeric code, comparing to pure python makes no sense, since people use numpy+numba to do that. Is julia much much faster than numba? I don't think so :)
Julia is generally in the same ballpark as Numba (depending on the application they should be within 2%). The difference is that Julia is a full language, while Numba breaks if you try to use it with anything else.
The more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy). It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language. I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.
This is one of the nicer aspects of Julia. It starts out being a great language to work in. Its easy to implement algorithms that are generally difficult in other languages. Its important to remember that most of the python ecosystem, isn't written in python. The functions are often thin wrappers/objects around the real computation, which is often written in a faster language, C/C++/Fortran. Julia excels in composabi…
Earlier quoted context omitted.
That’s a really good question and I’m not sure I fully understand all the reasons. One big one is that Java intentionally has made interop with native libraries quite difficult. One cannot take a language seriously for numerical computing if you can’t easily call BLAS, LAPACK and FTTW just to start, and there’s a neverending supply of such efficient native libraries. Julia, on the other hand makes it easy to write ef…
> I also think that the JVM object model is quite limiting for numerical computing. They still don’t support value types This is mostly true, but the primitives are value types and you can get some things done with them. (Not enough to make Java good for these use cases, no.) I.e. write float[] instead of Float[] and you have a contiguously allocated region of memory that can be efficiently accessed.
Neither language have proper tail call elimination, which, is absolutely insane to me. Yall really just write procedural code for everything?
Tail calls and while loops are essentially equivalent, so why care whether a language prefers one or the other?