Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia 1.4

github.com

Post reply on HN
Print view View original

101–110 of 137 posts

Re: Julia 1.4

#101

Post by eigenspace »

dklend122 wrote: post #99
eigenspace wrote: post #93

Earlier quoted context omitted.

Who is going to make the scientific ecosystem? Julia and Python's scientific ecosystems are so strong precicesly because they get domain experts in those ecosystems to write the software they need for their niche. Machine learning programmers aren't about remake DifferentialEquations.jl or scipy in Swift. I've yet to meet a single scientist from a field outside of machine learning who was seriously excited for swift.…

> Who is going to make the scientific ecosystem? Google and apple. Apple already is working on a swift-numerics package. Look at TF python and jax. They've re-implemented chunks of scipy and numpy twice, hired people to work on plotting (altair) etc And that's with python. Their engineering time will go much further with swift, obviously.

Google and Apple are not going to make a full on scientific ecosystem because they don't have the domain experts or the motive.

Numpy is not the same thing as scipy.

DifferentialEquations.jl in julia is a great example of what it actually takes to make a real, competitive differentiation equation library. The sort of stuff that was built there requires a deep connection to the scientific and mathematics literature. Cash won't cut it.

Another great example that'll resonate with physicists at least is things like ITensors.jl https://github.com/ITensor/ITensors.jl. Apple and Google are not going to make something like that.

Re: Julia 1.4

#102

Post by ddragon »

qchris wrote: post #97
ddragon wrote: post #84

Earlier quoted context omitted.

I wonder if the issue here is the graduate students part more than the Julia part (although obviously the tooling needs work). Maintainability is an art that comes with experience (or proper guidance), and I had to deal in college with a ton of really scary C code from other students (like all the program in the main function with 2000 lines scary). And in the same way in a professional scenario (in a polyglot compan…

I think that this is very insightful. At least in my experience, maintainable code isn't something that naturally arises. It comes from experience and interest in writing code that will need to be maintained. A lot of students (or academic researchers in general) don't really pay much attention to these things, which is totally understandable given their overall incentive structure. I'm working with a computer scienc…

Julia is definitely trickier to control because it's an extremely powerful language. After all you can write even programs that can rewrite itself in it (like Cassette/Zygote, which is more than simply AST manipulation but full IR manipulation), there is no limit to how clever your program can be. But that level of introspection in the future could bring amazing linter capability to the language, that could detect all kind of bad practices as you type (for example type piracy, type instabilities, and possibly enforce customizable policies) which will become important tools as more people aim to use the language in production environments.

And until then the best practices for larger project using Julia's paradigm (Multiple Dispatch, which is by itself not as well understood as OOP or functional) will probably become more largely known. I feel like Julia projects should not grow large not because it's not suitable for large scale projects, but because even large scale Julia projects should emerge from the composition of many small and maintainable Julia projects.

Re: Julia 1.4

#103

Post by dklend122 »

eigenspace wrote: post #101
dklend122 wrote: post #99

Earlier quoted context omitted.

> Who is going to make the scientific ecosystem? Google and apple. Apple already is working on a swift-numerics package. Look at TF python and jax. They've re-implemented chunks of scipy and numpy twice, hired people to work on plotting (altair) etc And that's with python. Their engineering time will go much further with swift, obviously.

Google and Apple are not going to make a full on scientific ecosystem because they don't have the domain experts or the motive. Numpy is not the same thing as scipy. DifferentialEquations.jl in julia is a great example of what it actually takes to make a real, competitive differentiation equation library. The sort of stuff that was built there requires a deep connection to the scientific and mathematics literature. C…

It's not JUST google and apple, I'm sure they have enough cash and expertise and will to create enough momentum to attract more domain experts in other areas. Especially once google brain and deepmind start working on things more complex than stacking layers, which is happening now.

In particular, do you have another example aside from DifferentialEquations.jl ?

Neural ODEs are hot enough that something like that could easily pop up in swift.

ITensors is interesting, but that's only one.

Re: Julia 1.4

#104

Post by dunefox »

tomerbd wrote: post #12
komuher wrote: post #9

Something about Julia from me. I was big Julia fan using it for last 1.5 year and in my company we are using Julia for data preprocessing and parsing hundred of GB's of data. (We also use Python for ML and start to moving julia code to Nim for data prepro) If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet. We have about 3.5k loc in…

I have to admit that over the past 20 years, java is like the most stable robust, huge project, multiple teams, IDE friendly, type safe, infinite tooling and libraries language of choice!

Type safe? NPE en masse.

IDE friendly? IDE support is only good because you absolutely need an IDE for this language.

It's an unwieldy, verbose mess of a language.

Re: Julia 1.4

#105

Post by systemvoltage »

johnsea wrote: post #95
systemvoltage wrote: post #66

Earlier quoted context omitted.

The negativity is warranted if you've ever run Julia or had to maintain it in production. Use it for your own Jupyter notebooks and personal analysis? Great! Need to debug some weird obscure error (which Julia does a poor job of reporting to the user, let alone which line in the stack trace) while having production pressure to get it up and running again? Julia is unquestionably, unarguably and utterly unsuitable. Do…

You probably cannot say which company this is? Or what production workflow? Or what you are using Julia for?

Unfortunately not because of obvious reasons, but we are a Biotech AI startup, about 180+ people.

Re: Julia 1.4

#106

Post by burntsushi »

systemvoltage wrote: post #70
burntsushi wrote: post #68

Earlier quoted context omitted.

Do you have any links to other material where I could read up on some of the problems folks have faced using Julia in production? Thanks!

I'd love to do a write up on the issues to guide the Julia team towards focusing on production and robustness, investing most of their efforts in a solid debugger support (Juno - totally incapable tool), and stopping all new features and "optimization" projects before getting these things sorted out. I am just too beaten, too bruised and injured to have the mental capacity to write an article. :( I am sorry if I am t…

Ah okay. Bummer. I wasn't necessarily asking you to write it, but maybe others have documented similar problems. I don't use Julia, but would love to here more detailed production environment experiences.

Re: Julia 1.4

#107

Post by johnsea »

systemvoltage wrote: post #105
johnsea wrote: post #95

Earlier quoted context omitted.

You probably cannot say which company this is? Or what production workflow? Or what you are using Julia for?

Unfortunately not because of obvious reasons, but we are a Biotech AI startup, about 180+ people.

Thank you. And: good luck!

Re: Julia 1.4

#108

Post by dunefox »

logicchains wrote: post #27
mbeex wrote: post #21

Earlier quoted context omitted.

> unless you implement your own Tensorflow OP in C++/Cuda I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I have some good experience with dlib, PyTorch's libtorch is on my todo list.

>I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I think Flux really excels when you're trying to do ML as "differentiable programming" (e.g. model-based reinforcement learning), because it can differentiate so much control flow via Zygote, which hooks into the compiler and differentiates the AST: https://github.com/FluxML/Zygote.jl

As an aside, what's the status of Flux and Zygote? Last time I looked they were integrating them and I got some errors.

Re: Julia 1.4

#109

Post by laughinghan »

systemvoltage wrote: post #73
laughinghan wrote: post #72

Earlier quoted context omitted.

Harshness is not the problem, credibility is. I don't mean that you sound like you're lying, but that it's impossible for anyone to evaluate your assertion or how it applies to their use case. Anyone who reads your comment (including me) just files it away in their head as "somebody on HN made an unsubstantiated comment that Julia sucks in production in some vague way". If I were in the position of making or advising…

You're right, I see the problem. I'll do an article and post it with detailed analysis. It will take some time because repos are like Tolstoy's quote - "All good repos are alike, every broken repo is broken in its own way". There are issues ranging from startup time, upstream packages breaking, constantly changing ecosystem (given for a new language), quirky behaviors and certain problems related to HTTP package and…

Interesting, those all sound like serious issues I would very much like to learn more about.

Re: Julia 1.4

#110

Post by dklend122 »

systemvoltage wrote: post #73
laughinghan wrote: post #72

Earlier quoted context omitted.

Harshness is not the problem, credibility is. I don't mean that you sound like you're lying, but that it's impossible for anyone to evaluate your assertion or how it applies to their use case. Anyone who reads your comment (including me) just files it away in their head as "somebody on HN made an unsubstantiated comment that Julia sucks in production in some vague way". If I were in the position of making or advising…

You're right, I see the problem. I'll do an article and post it with detailed analysis. It will take some time because repos are like Tolstoy's quote - "All good repos are alike, every broken repo is broken in its own way". There are issues ranging from startup time, upstream packages breaking, constantly changing ecosystem (given for a new language), quirky behaviors and certain problems related to HTTP package and…

I'd be interested in reading this as well. In the past legitimate critical blog posts have made a positive impact on the Julia community.
Post reply on HN