Live data from Hacker News

My Journey from R to Julia

drtomasaragon.github.io

111–120 of 120 posts

Re: My Journey from R to Julia

#111
post #51
post #31

Earlier quoted context omitted.

That's some pretty generic premature optimization cargo culting. If you have a huge data set and some understanding what you're doing, the bottlebecks will be pretty obvious.

One of the reasons why Mark Godbolt created compiler explorer was to prove teammates that what for them was pretty obvious actually wasn't.

In Julia I do not need the Godbolt compiler explorer. The macros `@code_llvm` and `@code_native` show me the LLVM IR and native code for a function.

  julia> @code_llvm debuginfo=:none 5.0 + 3
  define double @"julia_+_156"(double %0, i64 signext %1) #0 
  {
  top:
    %2 = sitofp i64 %1 to double
    %3 = fadd double %2, %0
    ret double %3
  }

  julia> @code_native debuginfo=:none 5.0 + 3
  ...
    vcvtsi2sd %rdi, %xmm1, %xmm1
    vaddsd %xmm0, %xmm1, %xmm0
    retq
  ...

Re: My Journey from R to Julia

#112
post #22

Earlier quoted context omitted.

One interesting thing is that if julia can prove what types a function will be called with at compile time, it doesn't have to do dynamic dispatch, so it has no overhead. It's what the julia folks call type-stable code

If ifs and buts were candy and nuts...

Now with better formatting...

This becomes an important part of optimizing Julia code. There is some tooling for this. Below, identity is a type stable function because we know that an Int64 argument results in a Int64 output. The macro code_warntype reveals the type analysis:

  julia> @code_warntype identity(5)
  MethodInstance for identity(::Int64)
    from identity(x) in Base at operators.jl:526
  Arguments
    #self#::Core.Const(identity)
    x::Int64
  Body::Int64
  1 ─     return x


  julia> f(x) = identity(x ≥ 0 ? x : x + 0.0) 
  f (generic function with 1 method)

  julia> @code_warntype f(5)
  MethodInstance for f(::Int64)
    from f(x) in Main at REPL[6]:1
  Arguments
    #self#::Core.Const(f)
    x::Int64
  Locals
    @_3::Union{Float64, Int64}
  Body::Union{Float64, Int64}
  1 ─ %1 = (x ≥ 0)::Bool
  └──      goto #3 if not %1
  2 ─      (@_3 = x)
  └──      goto #4
  3 ─      (@_3 = x + 0.0)
  4 ┄ %6 = @_3::Union{Float64, Int64}
  │   %7 = Main.identity(%6)::Union{Float64, Int64}
  └──      return %7

Re: My Journey from R to Julia

#113

This is a pretty weak article. The author lists five reasons an epidemiologist would be interested in Julia and then only gives a (kind of simple and contrived) example for one of them.

My father is a (retired) veterinary epidemiologist (i.e. he looked at spread of diseases in Animal populations) probably a function of his age but he wrote most of his models and simulations in Pascal and then switched to Basic eventually.

From my conversations with him the programming language was not the bottleneck for him it was the integration with GIS software and spatial mapping packages which caused him problems. A lot of programming languages did not mesh together very well with spatial mapping tools available to him at the time.

The first language I ever saw him use was Turbo Pascal, later he would use QBasic and GWBasic eventually he was using Visual Basic).

Towards the end of his career I believe he looked at other languages like Python and Java but I don't believe he found them very compelling. Python I believe has better spatial tools available now but it would have been relatively early in the languages life my Dad was looking at it and those packages probably did not exist.

Re: My Journey from R to Julia

#114
post #95

The article is supposed to tell us why Jilia is better than R, but it mainly focuses on one feature - multiple dispatch. Can someone please explain - does multiple dispatch provide any advantage over other function call strategies, and even if it does how much effort would it save, how much shorter or less ambiguous our code would become.

This isn't even showcasing what multiple dispatch is, and its power, see https://www.youtube.com/watch?v=kc9HwsxE1OY

Re: My Journey from R to Julia

#115

Interestingly, I found myself going the other way. Let me first say that R is a hilariously weird-feeling and janky language. The Julia features mentioned (structure are good for organising; compilation and better data structures mean you need to worry less about accidentally writing code that is 10x or 100x slower than it ought to be, which tends to matter a lot for interactive use) are definitely useful, and magica…

Take a look at DataFramesMeta for nicer manipulation of Julia's dataframes. Your example would look like

  julia> df = DataFrame(y = rand(10^6), filter=randn(10^6));

  julia> @transform!(df, :x = cumsum(:y .* :filter))
  1000000×3 DataFrame
       Row │ y          filter      x
           │ Float64    Float64     Float64
  ─────────┼────────────────────────────────────
         1 │ 0.0726663   1.7213       0.125081
         2 │ 0.183898   -0.392131     0.0529686
         3 │ 0.150274    1.08083      0.21539
      ⋮    │     ⋮          ⋮            ⋮
It's particularly nice in conjunction with @chain [1].

[1] https://juliadata.github.io/DataFramesMeta.jl/dev/#Chaining-...

Re: My Journey from R to Julia

#116
post #81
post #21

Earlier quoted context omitted.

The obsession with cpu speed almost always confuses me in these topics. Time it takes to program is way more important, and that’s where a terse language like R shines. The base/most common functions are almost always executing C anyway. It’s kind of like lisp in that it’s easy to write slow code, but who cares if it’s “fast enough”? Also, it’s almost always easy to speed up if necessary at the R level and R’s C API…

One of the key points of Julia is that the language you use for performance critical parts is also Julia. That applies to both the libraries like DataFrames.jl and for situations where you'd drop to a lower level language when optimising. I think being productive in Fortran or C++ is unrealistic for most scientific programmers.

[deleted]

Re: My Journey from R to Julia

#117
post #110

Earlier quoted context omitted.

Oh, well here's a post by Jeff Bezanson, since you've decided to just talk out of your ass... https://discourse.julialang.org/t/what-dont-you-like-about-j... Hope the uh... co-creator warrants enough merit as a source.

So the 2 year old post by Jeff on discourse says that breaking changes would have to go into an eventual 2.0. Whereas in the 6 month old keynote the same Jeff says there is no plan for breaking changes and hence no 2.0 is planned. So Julia will remain stable. And maybe we get interfaces (or rather traits) at some point in the 1.x release family.

Sure, if it happens, then I'll change my tune pretty quick.

I don't stay plugged in beyond reading the changelog whenever a new version comes out, so I didn't know that bit about the keynote. I've noticed as well that the Julia community produces an absolutely extraordinary amount of conference talks/video content, I don't have the time for the finer details.

I've also noticed a distinct and crucial lack of long-term vision for Julia from the co-founders. I've also read some dramatic (unverified) claims that I won't repeat here about one of them. Frankly, I think a better group of people could be assembled to steward the language and its ecosystem, but I don't see much chance of that happening. Julia exists in a weird little place where being terrible to read and maintain doesn't matter as long as it lives up to its promises of being very, very fast, in most of the applications it is used for. That's all well and good, but you don't build a solid foundation for a community to really depend on that tool for bedrock tasks.

I've been unfortunate enough to read some .jl code in '22, and it was dreadful. I truly don't understand how multiple dispatch makes anybody's life easier, it's an absolute nightmare of unmaintainable code that calls any of dozens or hundreds of methods, the performance of the entire application essentially dependent on whether or not that type is stable and of course, there's no way to know that without reading the definition of the unknown method that gets called.

Personally, I have my eyes peeled for github.com/exaloop/codon for higher performance stuff with Python. It's already an order of magnitude faster than pypy for most cases, and equally more usable for practical work than Julia, imo.

Anyhow, when it's all said and done, there's a lot of computing being done and a lot of money changing hands and so forth. All's well that ends well, despite never really being done well. /shrug

Re: My Journey from R to Julia

#118
post #110

Earlier quoted context omitted.

So the 2 year old post by Jeff on discourse says that breaking changes would have to go into an eventual 2.0. Whereas in the 6 month old keynote the same Jeff says there is no plan for breaking changes and hence no 2.0 is planned. So Julia will remain stable. And maybe we get interfaces (or rather traits) at some point in the 1.x release family.

Sure, if it happens, then I'll change my tune pretty quick. I don't stay plugged in beyond reading the changelog whenever a new version comes out, so I didn't know that bit about the keynote. I've noticed as well that the Julia community produces an absolutely extraordinary amount of conference talks/video content, I don't have the time for the finer details. I've also noticed a distinct and crucial lack of long-term…

> I don't stay plugged in beyond reading the changelog whenever a new version comes out ... > I've also noticed a distinct and crucial lack of long-term vision for Julia from the co-founders.

Don't you find those two statements contradictory? There's a pretty striking contrast between your claim to ignorance and your confidently sweeping generalization (sadly, those two often do come in pairs.)

> I've also read some dramatic (unverified) claims that I won't repeat here about one of them.

If this is what I think it is, the claims were not substantiated in any way (even though it would be easy), and seemed quite outlandish, frankly.

> I've been unfortunate enough to read some .jl code in '22, and it was dreadful. I truly don't understand how multiple dispatch makes anybody's life easier

It's really hard for me to understand this opinion, given that Julia code, to me, is far more appealing than all the most common alternatives. In particular, multiple dispatch is such an obviously natural paradigm, that it's just hard to fathom why everyone don't just 'get it' right away. I mean, not taking all input argument types into account now seems to me like a completely artificial, even perverse, restriction. Why?

I guess this is why people argue on in the internet.

Re: My Journey from R to Julia

#119
post #118

Earlier quoted context omitted.

Sure, if it happens, then I'll change my tune pretty quick. I don't stay plugged in beyond reading the changelog whenever a new version comes out, so I didn't know that bit about the keynote. I've noticed as well that the Julia community produces an absolutely extraordinary amount of conference talks/video content, I don't have the time for the finer details. I've also noticed a distinct and crucial lack of long-term…

> I don't stay plugged in beyond reading the changelog whenever a new version comes out ... > I've also noticed a distinct and crucial lack of long-term vision for Julia from the co-founders. Don't you find those two statements contradictory? There's a pretty striking contrast between your claim to ignorance and your confidently sweeping generalization (sadly, those two often do come in pairs.) > I've also read some…

Ah, well I should’ve clarified. I was very, very excited about the language from about 0.4 to 1.2ish.

In that time, I was nearly obsessive in reading all the news, though I wasn’t writing any jl code at all during that time. After 1.0, I used it for a few little things here and there in my business, some basic csv munging and a few other one-off tasks. It did fine, ofc.

The claims, yeah I mean, I don’t know the parties involved personally but because we are not far apart in the social graph and I don’t need the headache this close to retirement. I love my opinions very much, but not when they involve personal controversy.

Multiple dispatch is a write-only benefit. In my line of work, I might come across different machines, different memory setups, different instruction sets, different c compilers, different latency profiles, different spoken language as documentation, etc.

I’m not a mathematician, but I clean up and leave businesses with pragmatic, clean, maintainable code after the mathematicians do their thing and move on. For me, good code is about how specifically and how clearly I can communicate an idea so that the next person (who won’t know shit about shit) can add a feature without breaking something, change a deploy, fix a small bug, update documentation, etc, without breaking things. In that regard, Julia code reads like someone’s manuscript about the issue and not a program written by a programmer.

To you, that’s desired. To me, that’s my actual worst nightmare because it means the longest mental checklist between getting onsite and leaving a finished product and getting paid.

You mention not taking all argument types, it’s just so stereotypically mathematician of an assumption that the code will just work with whatever garbage you send it.

Because maybe that function needs to run on a 32bit system somewhere, and it will fail?

Maybe some of the other methods have a bug, and you won’t notice until you use the original method with a certain type, that you hadn’t considered? Julia can’t prove this sort of thing, to my knowledge.

I’m not saying python can, necessarily, but I know that there are very strong confidence levels and that if something borks, I can arrive at the scene of the crime within a few minutes of any given stacktrace and docker image, or whatever.

I don’t mean to argue. It’s clear we disagree. That’s fine. I just can’t take Julia seriously because they treat data like math on a whiteboard, and it’s just a ridiculous way to program a computer. Like I said, it gets a lot of computing done, and a lot of people are going home paid well. It’s all good, to that end.

Edit: I likely won’t reply further, it’s really annoying to scroll back in HN to see replies without linked notifications. Be well!

Re: My Journey from R to Julia

#120
post #118

Earlier quoted context omitted.

> I don't stay plugged in beyond reading the changelog whenever a new version comes out ... > I've also noticed a distinct and crucial lack of long-term vision for Julia from the co-founders. Don't you find those two statements contradictory? There's a pretty striking contrast between your claim to ignorance and your confidently sweeping generalization (sadly, those two often do come in pairs.) > I've also read some…

Ah, well I should’ve clarified. I was very, very excited about the language from about 0.4 to 1.2ish. In that time, I was nearly obsessive in reading all the news, though I wasn’t writing any jl code at all during that time. After 1.0, I used it for a few little things here and there in my business, some basic csv munging and a few other one-off tasks. It did fine, ofc. The claims, yeah I mean, I don’t know the parti…

> In that regard, Julia code reads like someone’s manuscript about the issue and not a program written by a programmer. To you, that’s desired. To me, that’s my actual worst nightmare

That's a pretty dubious attribution of intention, though, that I desire this. I want clean, maintainable code.

> You mention not taking all argument types, it’s just so stereotypically mathematician of an assumption that the code will just work with whatever garbage you send it.

But that's not what I said, and it's not what multiple dispatch means. You are talking about generic functions and duck typing -- essentially, code that has no type restrictions. Multiple dispatch means that types of all arguments are considered, and you are free to be as restrictive as you wish. You can specifically and concretely type every single input, and probably make the code much more predictable and to your liking.

The amount of genericity vs type safety is a trade-off between different advantages, and you have a lot of freedom to choose.

Post reply on HN