Live data from Hacker News

Why We Use Julia, 10 Years Later

julialang.org

111–120 of 144 posts

Re: Why We Use Julia, 10 Years Later

#111
post #47

Earlier quoted context omitted.

For which kinds of problems are matlab and python unworkable?

All of these have some exceptions, but are for the most part true: high energy physics lots of computational bio fluid dynamics rendering In general, python and matlab really struggle in problems where you want maximum performance, but the most efficient algorithms aren't vectorized. In some cases, this is solvable by writing python libraries in C/C++, but especially in scientific fields, the end users are often the…

high energy physics (HEP) shameless plug: https://github.com/tamasgal/UnROOT.jl

indeed, the grants (in the millions $) given to rewrite C++ or Python just to handle array (because for loop sucks) and ends up making monorepo blobs is jaw dropping -- while it's almost free in Julia, with 1/100 of the line of code and more flexibility we can match and beat even C++ code...

Re: Why We Use Julia, 10 Years Later

#112
post #97

Earlier quoted context omitted.

I’ve become a big fan of Numba. No more awkward, human-unreadable vectorized code.

IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.

I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba.

Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ecosystem from within my vectorized code either.

> The code is fast, but you lose the ability to organize your data the way you would in regular python,

Can you clarify what you mean by this? I lose the ability to organize my data the way I'd like if I'm vectorizing my code too.

Re: Why We Use Julia, 10 Years Later

#113

Earlier quoted context omitted.

> immutable struct with multiple fields, many of which have defaults With `Base.@kwdef` (or the enhanced `@with_kw` from Parameters.jl) on the struct definition, that example can be: m = App(name = "My program", author = "Me, me@mail.com", version = "1.0.2", about = "Explains in brief what the program does", arg = index(Arg("in_file"), 1) which seems pretty nice to me. Also, there's Chain.jl for a better pipe operato…

That uses keyword arguments though which is bad for performance, which is the main reason I'd like to use immutable structs.

Unless you have some very weird use case (and probably even then), that sounds like a bad application of the "keyword arguments bad for performance" mental model.

The reason to use immutable structs is better performance when you use them, not when you construct them - how often are you constructing new struct objects that the constructor performance is a significant issue? And if you are constructing many many thousands of objects to the level it becomes a performance concern, any difference from keyword arguments will be easily dwarfed by the cost of actually constructing the struct in memory.

Also, these constructors ar simply passing on these named arguments to the non-keyword constructors. Anything complex you're doing during construction will be happening in the non-keyword constructors, and those do get specialized to have good performance; these keyword constructors are simply providing a small, simple interface to them, in a way that shouldn't affect performance in any measurable way.

Re: Why We Use Julia, 10 Years Later

#114
post #112

Earlier quoted context omitted.

IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.

I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba. Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ec…

I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.

Re: Why We Use Julia, 10 Years Later

#115
post #97

Earlier quoted context omitted.

I’ve become a big fan of Numba. No more awkward, human-unreadable vectorized code.

IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.

Not true at all for me, because in my situation there are some functions which have tight loops that take 99% or more of the execution time. Put the numba decorator in front of them and the code is 100+X faster. Clearly it's not that simple for everyone. But for some people, it can be. So it doesn't make sense to call it a panacea, and it also doesn't make sense to say it doesn't work. It works for some people, not for others.

Re: Why We Use Julia, 10 Years Later

#116

Earlier quoted context omitted.

Any reference to how Julia outperform Fortran in BLAS? I thought it is/was calling existing BLAS libraries?

Julia currently ships with fortran based Blas, but Octavian.jl is apure Julia matmul that is faster. (it's nowhere near finished though)

And the "how" behind Octavian.jl is basically LoopVectorization.jl [1], which helps make optimal use of your CPU's SIMD instructions.

Currently there can some nontrivial compilation latency with this approach, but since LV ultimately emits custom LLVM it's actually perfectly compatible with StaticCompiler.jl [2] following Mason's rewrite, so stay tuned on that front.

[1] https://github.com/JuliaSIMD/LoopVectorization.jl

[2] https://github.com/tshort/StaticCompiler.jl

Re: Why We Use Julia, 10 Years Later

#117
post #92

I really, really wanted to like Julia. The syntax looked perfect to me (coming from scheme), and somehow everything felt 'right'. That's when I discovered that compiling to binary seemed to be frowned upon. I found some documentation on how to do this back when 1.0 came out, but it looked unnecessarily complicated and 'third-party'. I'm guessing Julia really is for sci-comp and data scientists, and not for producing…

I don't think it's frowned upon to compile, many people want this capability as well. If you had a program that could be proven to use no dynamic dispatch it would probably be feasible to compile it as a static binary. But as long as you have a tiny bit of dynamic behavior, you need the Julia runtime so currently a binary will be very large, with lots of theoretically unnecessary libraries bundled into it. There are…

Yeah, not frowned upon at all. StaticCompiler.jl [1] has made huge strides in the past few weeks actually thanks to Mason and Valentin (swapping in GPUCompiler as the backend); you can even compile to a tiny standalone binary without linking to the runtime if you're willing to use some tricks e.g. [2] to avoid GC allocations (stack allocations and manual heap allocations are both fine):

    # This is all StaticCompiler-friendly
    using StaticTools

    function print_args(argc::Int, argv::Ptr{Ptr{UInt8}})
        # c"..." lets you construct statically-sized, stack allocated `StaticString`s
        # We also have m"..." and MallocString if you want the same thing but on the heap
        printf(c"Argument count is %d:\n", argc)
        for i=1:argc
            # iᵗʰ input argument string
            pᵢ = unsafe_load(argv, i) # Get pointer
            strᵢ = MallocString(pᵢ) # Can wrap to get high-level interface
            println(strᵢ)
            # No need to `free` since we didn't allocate this memory
        end
        println(c"That was fun, see you next time!")
        return 0
    end

    # Compile executable
    using StaticCompiler # `] add https://github.com/tshort/StaticCompiler.jl` to get latest master
    filepath = compile_executable(print_args, (Int64, Ptr{Ptr{UInt8}}), "./")
yielding:

    shell> ./print_args 1 2 3 4 5.0 foo
    Argument count is 7:
    ./print_args
    1
    2
    3
    4
    5.0
    foo
    That was fun, see you next time!

    shell> hyperfine './print_args hello there'
    Benchmark 1: ./print_args hello there
      Time (mean ± σ):       2.2 ms ±   0.5 ms    [User: 0.8 ms, System: 0.0 ms]
      Range (min … max):     1.5 ms …   5.5 ms    564 runs

      Warning: Command took less than 5 ms to complete. Results might be inaccurate.

    shell> ls -lh print_args
      -rwxr-xr-x  1 user  staff   8.5K Feb 10 02:36 print_args

GC allocations are allowed if you use instead the approach in this PR [3], but we haven't wrangled that approach to produce standalone binaries yet.

[1] https://github.com/tshort/StaticCompiler.jl

[2] https://github.com/brenhinkeller/StaticTools.jl

[3] https://github.com/tshort/StaticCompiler.jl/pull/58

Re: Why We Use Julia, 10 Years Later

#118
post #112

Earlier quoted context omitted.

I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba. Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ec…

I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.

This doesn't really make sense. If you don't care about performance then you have no use for Numba.

Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.

Re: Why We Use Julia, 10 Years Later

#119
post #118

Earlier quoted context omitted.

I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.

This doesn't really make sense. If you don't care about performance then you have no use for Numba. Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.

I think the point is that one wants to write code that is similar to regular non-performance sensitive python code, with classes and everything, and still have it be fast.

Re: Why We Use Julia, 10 Years Later

#120
post #118

Earlier quoted context omitted.

I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.

This doesn't really make sense. If you don't care about performance then you have no use for Numba. Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.

> Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.

The comment you replied to explicitly says "regular python".

Post reply on HN