Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia 1.6: what has changed since Julia 1.0?

oxinabox.net

Post reply on HN
Print view View original

71–80 of 212 posts

Re: Julia 1.6: what has changed since Julia 1.0?

#71

Post by saiojd »

Sukera wrote: post #32
saiojd wrote: post #26

Earlier quoted context omitted.

A few months ago. First time to plot is noticeably better than it was a few versions ago, but still extremely slow compared to Python. The article gives a benchmark of 9 seconds. I mean, come on. The main problem I had was simply that what any time I needed to modify a struct field, or anytime my program crashed, or any time the buggy IDE extension crashed, I needed to recompile everything. I also haven't found anyon…

I know that there are plans to cache even more, but other than that I can basically only recommend to put distinct projects into proper projects (with a module and Project.toml). That will already cache precompiled code for that module, even between sessions. Having things in a script won't have that benefit. For experimenting with struct layouts, I've found that NamedTuples ( https://docs.julialang.org/en/v1/base/ba…

I've read a bit on type invalidation and I know it's a hard problem (in fact it's hard for me to even wrap my head around it, lol). Still, it's unfortunate. One thing I would like to know is if the difficulties with invalidation are a symptom of the dynamic semantics, or of the compilation model.

Namedtuples are cool, but I'm not sure I understand the tradeoffs between using them and using structs. Can I just replace all structs in my project with named tuples, without having a performance hit?

Re: Julia 1.6: what has changed since Julia 1.0?

#72

Post by saiojd »

DNF2 wrote: post #46
cmeacham98 wrote: post #41

Earlier quoted context omitted.

You seem to be falling into the same trap you lament in your comment: - The other side is "these people" that "won't stop" - They're striving for "hyper-efficiency" at the cost of "hundreds and thousands of hours" - People who complain about this issue don't (or do significantly less of) reading documentation/code/etc People talk about TTFP because it is a real issue that is off-putting for many programmers that woul…

I cannot fathom how ttfp is important. It's time to first plot, not to every plot. After it's down to ~10 seconds, why on earth does anyone care?

You can't possibly be serious. As things stand, even just redefining a struct field requires restarting the interpreter, which forces you to wait ~10-30 seconds for everything to compile, over and over again. Maybe you only use Julia for small scripts, and can afford to never restart the interpreter?

Re: Julia 1.6: what has changed since Julia 1.0?

#73

Post by systemvoltage »

eigenspace wrote: post #65
systemvoltage wrote: post #52

Earlier quoted context omitted.

Do you think R or Matlab is a general purpopse language? Sure you could do all these things but should you? Julia is clearly positioned as a scientific computing language. Let's be clear.

Doing these things in julia is very different from doing these things in R or Matlab. The tooling and ecosystem for non-scientific applications in Julia is growing rapidly and is quite competent. Julia is absolutely a general purpose language. It’s user base skews heavily towards scientific computing, but the demographics and ecosystem are broadening daily.

Having used Julia for 2+ years, I couldn't disagree more. Productionizing Julia code has been a total nightmare. The community library support has been growing but hasn't gone through the wringer. Just because things are improving doesn't provide a meaningful understanding against its competitors.

I don't see any reason to use Julia over Go for backend webservers. Rust or C++ for systems programming. And frankly, I prefer Python for scientific computing.

Julia also has a tiny standard library and lots of flaky external libs which make productionization of code a risky adventure which I have personally been bitten by.

Most people are allured by Julia's overhyped marketing which is a shame because the original paper by Stefan is pretty impressive. We're seeing some criticisms of Julia in this thread, rightfully so.

My advise to people who are subscribed to Julia's marketing is to listen to people that are complaining. No one wants to just complain, they're saying that because of many reasons. Be humble and try to listen, accept Julia's many shortcomings (error messages and stack traces, library support, startup time, IDE, debugging, etc.). Julia has many shortcomings that are only apparently after using it outside of the Jupyter Notebooks. Not accepting those makes you an annoying fanboy.

Re: Julia 1.6: what has changed since Julia 1.0?

#74

Post by phillc73 »

baldfat wrote: post #29
phillc73 wrote: post #24

Earlier quoted context omitted.

Have you tried Query.jl or DataFramesMeta.jl?

I don't like working with DATA TABLES UNLESS it is a HUGE data frames. Then if it is huge I'll go towards sparks. I normally am working with under a million objects which with today's computers is not that big. Edit I meant to say that DATA TABLES library in R reminds me more of Query.jl then tiddyverse

Query.jl supports two different paradigms, one inspired directly by LINQ, the other by dplyr.[1] I actually prefer data.table in R, over dplyr, and Query.jl is really quite different.

[1] http://www.queryverse.org/Query.jl/stable/

Re: Julia 1.6: what has changed since Julia 1.0?

#75

Post by enriquto »

tomrod wrote: post #6

I come back every so often to check out Julia again. I have hopes for it. Some questions still in mind since I reviewed previously (1) How is its database connectivity? (2) Is there something like python's `requests` lib? (3) Are the features mature enough that I don't anticipate major rewrites for code each year?

> (3) Are the features mature enough that I don't anticipate major rewrites for code each year?

There is a very good heuristic for that. Write a non-trivial program using the last version of the language and according to current conventions. Then look how far can you go into past versions of the language so that your program runs correctly.

If the oldest version of the language that runs your program is X years old, then you can expect your program to stop running after X years in the future.

Re: Julia 1.6: what has changed since Julia 1.0?

#76

Post by Certhas »

CyberDildonics wrote: post #58
oxinabox wrote: post #23

Earlier quoted context omitted.

> You don't need to compile the plotting library every time. You don't need to but Julia does. Its definately a issue, and I know it is being worked on. Julia doesn't store compiled binary code between sessions. Unless you compile it into a sysimage. There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice.

> There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice. I put a lot of hours into julia five years ago and all the same things were being said. I don't know why the compilation is so slow or why the caching is so bad, but it was the main complaint then and still is. The solutions are all 'just around the corner'. It remind…

I don't think that's fair, simply because things have actually gotten a lot better. This is shown in this blog post and it also is evident to people who use Julia:

https://i.redd.it/ik4uymvb28k51.png

Also, compiling a sysimage used to be an arcane art, and now works reasonably simply/well. It's easy to imagine a future where, with the tooling that is already there and without a magic breakthrough in caching Julia code, we simply get a per project sys-image in VSC that is recompiled when needed.

Re: Julia 1.6: what has changed since Julia 1.0?

#77

Post by DNF2 »

CyberDildonics wrote: post #58
oxinabox wrote: post #23

Earlier quoted context omitted.

> You don't need to compile the plotting library every time. You don't need to but Julia does. Its definately a issue, and I know it is being worked on. Julia doesn't store compiled binary code between sessions. Unless you compile it into a sysimage. There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice.

> There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice. I put a lot of hours into julia five years ago and all the same things were being said. I don't know why the compilation is so slow or why the caching is so bad, but it was the main complaint then and still is. The solutions are all 'just around the corner'. It remind…

Well, a lot of resources have been put into reducing compilation times, and large improvements have been achieved. It's not just perennially 'around the corner', the improvements are tangible and happening right now.

Re: Julia 1.6: what has changed since Julia 1.0?

#78

Post by ku-man »

eigenspace wrote: post #31
kitsune_ wrote: post #17

It takes 180s to import Statistics, PlutoUI, Images and OffsetArrays in Pluto on this Macbook Pro with 32GB that is a couple of years old, while the fans are going crazy. If I'm unlucky a worker process will seg fault. All in all the interactive / REPL part (even with stuff like Revise) is kind of a let down thanks to the slow precompilation / lack of caching? I like it, but I really don't see how it is ever going to…

I assume that's including precompilation, otherwise something is seriously wrong with your computer. Precompilation can take a while, but it's cached between sessions, so it should be much faster the second time. Also, in version 1.6, we have multithreaded precompilation which happens at installation time instead of when you first try to load the package. This makes it much faster.

Julia people say "in version 1.6... precompilation... is much faster". But they said the same when version 1.2 was released, same when 1.3, same with 1.4, and with 1.5, do you see a trend here?. It's clear the precompilation issue was never solved, otherwise it wouldn't be "fixed" in every single release.

Re: Julia 1.6: what has changed since Julia 1.0?

#79

Post by superbcarrot »

saiojd wrote: post #16

I've tried Julia and really liked it, but the user experience was pretty bad. The language really needs faster interactivity or a strong type checker. I found myself waiting after than compiler a lot more than in some AOT compiled language like Rust... I fear the language suffers from being overused by people who are familiar with Matlab and Python and draws too much inspiration from them, much like how Rust draws to…

I'm trying really hard to like Julia but constantly chasing "ERROR: MethodError: no method matching" messages gets frustrating very quickly.

Re: Julia 1.6: what has changed since Julia 1.0?

#80

Post by lhn »

CyberDildonics wrote: post #58
oxinabox wrote: post #23

Earlier quoted context omitted.

> You don't need to compile the plotting library every time. You don't need to but Julia does. Its definately a issue, and I know it is being worked on. Julia doesn't store compiled binary code between sessions. Unless you compile it into a sysimage. There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice.

> There are apparently reasons why caching compiled binary like this is complicated in Julia. But once that is solved, wow things are going to be nice. I put a lot of hours into julia five years ago and all the same things were being said. I don't know why the compilation is so slow or why the caching is so bad, but it was the main complaint then and still is. The solutions are all 'just around the corner'. It remind…

I don't think it's fair to say Julia's problems haven't changed at all. While compilation latency is still an ongoing issue, it has consistently and noticeably improved over the years. Package caching is much better, you can save compilation results you depend on in your workflow with PackageCompiler, etc. There are now incredible tools (e.g., SnoopCompile.jl) for package developers to inspect closely where the compiler might have difficulty and fix the issues.

The major source of improvements in 1.6 is eliminating method invalidations. Julia's flexibility makes it vulnerable to invalidating already compiled code as new packages are loaded and new methods are defined. This triggers a disastrous cascade of recompiling a bunch of things, and is the main conceptual reason why nothing lower than type-inferred code is cached. If your method will be recompiled anyway, then what good is it to save the native code in the first place? Now that invalidations can be efficiently diagnosed and patched, there is definitely interest into caching lower levels of code in the compilation process, potentially even native machine code.

All of these progress however requires labor and care. I'd say the Julia community has spent an admirable amount of efforts into its latency issue, but there's a limit to how fast you can address these problems through open-source development without backing from major tech companies. Imagine the improvements to the Julia compiler had Google chose Julia for its S4TF project, for instance.

Post reply on HN