Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia 1.6 addresses latency issues

lwn.net

Post reply on HN
Print view View original

81–90 of 160 posts

Re: Julia 1.6 addresses latency issues

#81

Post by ChrisRackauckas »

smabie wrote: post #78
adgjlsfhk1 wrote: post #72

Earlier quoted context omitted.

Anything in specific you feel like is missing from the module system and testing framework? Knowing what features people want helps a lot with setting development priorities.

It seemed to me that no one actually made projects that used the module system at all? Like no name spacing what so ever, just a bunch of include!s

Every package is a module, and Julia is well-known to not have monorepos and instead split packages into small modules. So I don't quite see how this follows: there's no PyTorch, instead there's Flux, NNLib, NNLibCUDA, NNLibAMDGPU, CUDA, GPUCompiler, KernalAbstractions, ... (it keeps going), all of which are documented packages in their own modules that make up the ML stack.

Re: Julia 1.6 addresses latency issues

#82

Post by ChrisRackauckas »

reikonomusha wrote: post #30
krastanov wrote: post #27

Earlier quoted context omitted.

You can already bundle pre-compiled plotting code in your Julia sys-image if you want. But Julia is not a plotting tool so it would be ridiculous to optimize it just for plotting. I want ODE solvers to have less latency, should I start expecting gnuplot to have built-in ODE solvers or the official installer of Julia to have the ODE libraries pre-compiled? Maybe this example would make it clearer: why does your argume…

I gave a proof-of-concept argument as to why something doesn’t need to take as long straight out of the box with no customization. Python is doing it sub-1s. You can also include a non-optimizing interpreter. My point is that being a general purpose language doesn’t inherently limit you in any way; instead it’s one’s choice of implementation strategy. Another strategy: when a user installs Julia, they select “fast-lo…

>Another strategy: when a user installs Julia, they select “fast-loading” libraries. You’d be surprise how small changes in UI/UX make huge perceived differences in quality and performance. I bet “Julia can already do this” too, but nobody does it because it’s not idiomatic and it’s not recommended up front.

Don't most people already use a bit of PackageCompiler.jl in their workflows? Maybe it just need to be mentioned more in introduction videos, but it's literally one line of code to do this and I don't see how that isn't the solution already. Most users I know have been doing it for a few years now.

For reference, with Plots and DifferentialEquations the command is:

using PackageCompiler; create_sysimage([:Plots,:DifferentialEquations])

It's simple enough to add to every tutorial. I think the issue is that intro videos just need to be updated.

Re: Julia 1.6 addresses latency issues

#83

Post by orbots »

dklend122 wrote: post #76
orbots wrote: post #70

Earlier quoted context omitted.

I'm a Julia user who has also given up as a Julia community member. The bubble is real. I found too often when I wanted to do something outside of that bubble and asked for help, I got unhelpful advice which amounted to "why do you want do something outside of our bubble?"

Can you give some examples?

One example. Not a very important one, but illustrative.

https://discourse.julialang.org/t/why-no-base-iterators-map/...

The "bubble effect" here was assumption that everyone has internet access and dependencies to external ( community maintained ) packages are zero cost.

Happily 2 years later this was resolved to my satisfaction.

Re: Julia 1.6 addresses latency issues

#84

Post by tbenst »

smabie wrote: post #51

So I used to be a big proponent of Julia, and in some ways, I still am. But I very recently tried to write a high performance production system in it, and was sorely disappointed. The tooling is just so buggy and it's clear that the community isn't really interested in using it for anything besides modeling/research in a Jupyter notebook. Things that kind of suck about using Julia for production: 1. Never could get R…

I use Julia as my main driver these days and have shared some of this experience but not all

1) I use VScode and have had 0 problems with Revise. It Just Works when using the Julia extension + built in REPL. I actually prefer the Julia environment to Python in VScode, I have way fewer problems when doing a Notebook-like workflow where I’m writing library code at the same time

2) Agreed, I also wish there was a better story here. I’m constantly frustrated by how little static support there is. It’s getting better though, eg precompilation in 1.6.

3) I haven’t had any issues here outside of the occasional update to VScode Julia extension that botches things

4) I’ve had quite a lot of luck on discourse and GitHub issues, as well as slack for the occasional small question

Rust is great for use cases where you have specific resource management needs, although seems an odd choice for general purpose scientific computing.

Re: Julia 1.6 addresses latency issues

#85

Post by xvilka »

smabie wrote: post #51

So I used to be a big proponent of Julia, and in some ways, I still am. But I very recently tried to write a high performance production system in it, and was sorely disappointed. The tooling is just so buggy and it's clear that the community isn't really interested in using it for anything besides modeling/research in a Jupyter notebook. Things that kind of suck about using Julia for production: 1. Never could get R…

> after some consideration of OCaml, but unfortunately the multi-core story still isn't there yet

It is supposed to land in the release after 4.13, which is the next one.

Regarding the scientific computations library there is Owl[1][2] which now has an almost finished book[3].

[1] https://ocaml.xyz/

[2] https://github.com/owlbarn/owl

[3] https://ocaml.xyz/book/

Re: Julia 1.6 addresses latency issues

#86

Post by neolog »

tbenst wrote: post #84
smabie wrote: post #51

So I used to be a big proponent of Julia, and in some ways, I still am. But I very recently tried to write a high performance production system in it, and was sorely disappointed. The tooling is just so buggy and it's clear that the community isn't really interested in using it for anything besides modeling/research in a Jupyter notebook. Things that kind of suck about using Julia for production: 1. Never could get R…

I use Julia as my main driver these days and have shared some of this experience but not all 1) I use VScode and have had 0 problems with Revise. It Just Works when using the Julia extension + built in REPL. I actually prefer the Julia environment to Python in VScode, I have way fewer problems when doing a Notebook-like workflow where I’m writing library code at the same time 2) Agreed, I also wish there was a better…

> had 0 problems with Revise

Try changing a struct.

Re: Julia 1.6 addresses latency issues

#87

Post by adgjlsfhk1 »

reikonomusha wrote: post #30
krastanov wrote: post #27

Earlier quoted context omitted.

You can already bundle pre-compiled plotting code in your Julia sys-image if you want. But Julia is not a plotting tool so it would be ridiculous to optimize it just for plotting. I want ODE solvers to have less latency, should I start expecting gnuplot to have built-in ODE solvers or the official installer of Julia to have the ODE libraries pre-compiled? Maybe this example would make it clearer: why does your argume…

I gave a proof-of-concept argument as to why something doesn’t need to take as long straight out of the box with no customization. Python is doing it sub-1s. You can also include a non-optimizing interpreter. My point is that being a general purpose language doesn’t inherently limit you in any way; instead it’s one’s choice of implementation strategy. Another strategy: when a user installs Julia, they select “fast-lo…

One thing that would help a lot is if julialang.org offered downloads with a bunch of common packages baked into a sysimage (modeled roughly of of anaconda). My rough estimate of packages to include would be Plots, BandedMatrices, DifferentialEquations, CSV, DataFrames, Flux, PyCall, (and probably some others). Having a distribution with a lot of the heavy hitter built in would make workflows using those packages much snappier.

Re: Julia 1.6 addresses latency issues

#88

Post by skybrian »

krastanov wrote: post #80
smabie wrote: post #78

Earlier quoted context omitted.

It seemed to me that no one actually made projects that used the module system at all? Like no name spacing what so ever, just a bunch of include!s

I feel like the DifferentialEquations package is a screamingly obvious counterexample to your claim. It has many sub-modules. Moreover, the whole point of multimethods is to live in the main semantic namespace, you do not need as many name spaces because there are no names to clash.

I've only kicked the tires on Julia a bit, but it seems like while methods are less likely to clash, they still might if you happen to use the same name and argument types? Or maybe unexpected method resolution causes a bug?

Re: Julia 1.6 addresses latency issues

#89

Post by neolog »

adgjlsfhk1 wrote: post #72
Tainnor wrote: post #60

Earlier quoted context omitted.

I would also add: 5. The module system is very primitive. 6. The testing framework is extremely barebones. I agree with your assessment, Julia is great for crunching numbers etc., but I wouldn't write a whole application in it.

Anything in specific you feel like is missing from the module system and testing framework? Knowing what features people want helps a lot with setting development priorities.

Modules could have signatures and be parametrized.

https://wiki.haskell.org/Module_signature

https://ocaml.org/learn/tutorials/modules.html

Re: Julia 1.6 addresses latency issues

#90

Post by lmm »

Buttons840 wrote: post #4

I'm a big fan of Julia. It does live up to its speed claims. I've implemented the board game Go in Python, Rust, and Julia and Julia is definitely closer to Rust in speed. Same algorithms were used for all implementations. Julia's time to first plot still has some problems. The Plots library can build animations, but the time to first animation on my computer is like 10 minutes, and the time to second animation is an…

> I'm a big fan of Julia. It does live up to its speed claims. I've implemented the board game Go in Python, Rust, and Julia and Julia is definitely closer to Rust in speed. Same algorithms were used for all implementations.

"Closer to Rust than to Python" is a wide range. Almost any non-scripting language (e.g. Java, OCaml, Haskell, Dylan...) would qualify, and that's normally not enough to give them a reputation as "fast".

Post reply on HN