Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Think Julia: How to Think Like a Computer Scientist

benlauwens.github.io

Post reply on HN
Print view View original

21–30 of 74 posts

Re: Think Julia: How to Think Like a Computer Scientist

#21

Post by babahoyo »

kirillseva wrote: post #16
babahoyo wrote: post #14

Earlier quoted context omitted.

Julia is increasingly becoming better than R and Stata for data cleaning. Many of its metaprogramming tools beat `dplyr` in syntax and features. So if the data-cleaning to regression stack (which i would guess is different than scientific computing) is your thing, then i would recommend trying Julia out.

Could you give some examples of how dplyr-based data cleaning code would look in modern julia?

Check out DataFramesMeta, which unfortunately isn't working on 1.0 yet. They have basically a 1-1 matching of `dplyr` verbs to julia versions.

I don't think a standardized and idiomatic data-cleaning process has been established yet, which is for the best right now. There is `JuliaDBMeta` for metaprogramming with JuliaDB tables, and the `Queryverse` for working with a wide array of objects.

One way that Julia's metaprogramming shines is with the ability to go into the AST and replace symbols, enabling local scopes that are more readable than other scopes. One workflow I'm excited to experiment with is something like this

    @as my_long_dataset d begin # make d = my_long_dataset in this scope
    @with d begin 
    t = :x1 + :x2 + x3 # these symbols are arrays inside this @with scope
    d.new_var = t # assign the variable
    end
    end
Of course, with the `@as` macro you probably don't save that many keystrokes if you are just doing `d.x` or `d[:x1, :x2]`... The ecosystem is still evolving but the point is that I like how you can replicate something like `attach` scoping in R without all the headaches. I think it makes a cleaning script feel more like you are only working with the data you care about.

Re: Think Julia: How to Think Like a Computer Scientist

#22

Post by zzleeper »

babahoyo wrote: post #14
cuchoi wrote: post #3

If I don't care about parallelism nor speed, is there a reason to learn Julia?

Julia is increasingly becoming better than R and Stata for data cleaning. Many of its metaprogramming tools beat `dplyr` in syntax and features. So if the data-cleaning to regression stack (which i would guess is different than scientific computing) is your thing, then i would recommend trying Julia out.

Early on (as a heavy Stata and Python user) I tried Julia and got quite discouraged by its messy treatment of missing values (and weights, etc). I've also tried R but also found lots of inconsistencies, so not enough reason to switch, besides when plotting nice graphs.

But I would say Julia is increasingly getting there. Comparable packages are WAY easier to write in Julia than in Stata/Mata, while being faster, so any gaps will keep disappearing in the next hears.

Re: Think Julia: How to Think Like a Computer Scientist

#23

Post by ms013 »

thatcat wrote: post #18
cjhanks wrote: post #12

Earlier quoted context omitted.

As somebody who appreciates Julia, my opinion is - probably not. That is, unless you want the opportunity to create a killer library that shows how the language features map well to other domains. But, I think a lot of people in scientific computing are tired of the typeless mess that Python/Numpy/Scipy code-bases evolve to be. And for those people, I think it has a lot of merit. At the end of the day, the language w…

What do you mean by production environment in the context of academia / R&D?

Usually “production” in those contexts means running on a cluster or supercomputer somewhere. That, or code being run on a machine not controlled by the developer by users who do not write code but just run a pre-existing program with different inputs. That’s how I (and colleagues) typically use the term ‘production’ in the academic/research environment that I work in. The issue with things like MATLAB in production is the lack of a license on the machine you want to run on - often people have it on their workstation, but if you get time at a supercomputing center or department cluster, odds are your local license doesn’t translate over to the system you want to run on.

Re: Think Julia: How to Think Like a Computer Scientist

#24

Post by fusiongyro »

cuchoi wrote: post #3

If I don't care about parallelism nor speed, is there a reason to learn Julia?

I've only been learning for about a week, but I think if you're a nerd for language design, you will appreciate it on an aesthetic level as a very tight design around a powerful concept. Common Lisp also has multiple dispatch, but I feel the integration of it into all the nooks and crannies of Julia really pays off. Julia's performance doesn't appear as a side effect of building on the LLVM or because they over-optimized the core, as it does in many young performance-oriented languages. Instead, it appears as a tangible benefit of a multiple-dispatch oriented design that makes it easy to add information to the system to improve performance without compromising the clarity of a sketch.

I have often felt that there are many discontinuities between "pretty" Haskell as it is taught and pragmatic Haskell. I haven't used Julia enough in anger to say it for sure, but I see in the way it works great potential for the pragmatic code to be as beautiful as the high-level and abstract code.

For a long time I have felt that Haskell represented the most mathematical language. Julia really shows that there are other ways of building a mathematical language with taste and style. It's oriented to practitioners and applied math folks rather than computer scientists and pure mathematicians. I have enjoyed seeing the differences between these systems quite a bit, and I think Julia has a bright future as a practical, daily-use system for science.

Re: Think Julia: How to Think Like a Computer Scientist

#25

Post by fusiongyro »

mlthoughts2018 wrote: post #20
cjhanks wrote: post #12

Earlier quoted context omitted.

As somebody who appreciates Julia, my opinion is - probably not. That is, unless you want the opportunity to create a killer library that shows how the language features map well to other domains. But, I think a lot of people in scientific computing are tired of the typeless mess that Python/Numpy/Scipy code-bases evolve to be. And for those people, I think it has a lot of merit. At the end of the day, the language w…

Having spent years working with numpy and Cython, then switching to Scala for years as well, I much prefer dynamic typing. Strong type safety is mostly just a waste of time.

One thing that makes Julia cool is that adding the types improves performance, but you don't have to add them if you don't want to.

Re: Think Julia: How to Think Like a Computer Scientist

#26

Post by skybrian »

zzleeper wrote: post #22
babahoyo wrote: post #14

Earlier quoted context omitted.

Julia is increasingly becoming better than R and Stata for data cleaning. Many of its metaprogramming tools beat `dplyr` in syntax and features. So if the data-cleaning to regression stack (which i would guess is different than scientific computing) is your thing, then i would recommend trying Julia out.

Early on (as a heavy Stata and Python user) I tried Julia and got quite discouraged by its messy treatment of missing values (and weights, etc). I've also tried R but also found lots of inconsistencies, so not enough reason to switch, besides when plotting nice graphs. But I would say Julia is increasingly getting there. Comparable packages are WAY easier to write in Julia than in Stata/Mata, while being faster, so a…

Did you see how Julia does missing values now?

https://julialang.org/blog/2018/06/missing

Re: Think Julia: How to Think Like a Computer Scientist

#27

Post by kevmo314 »

gopalv wrote: post #17
cuchoi wrote: post #3

If I don't care about parallelism nor speed, is there a reason to learn Julia?

> is there a reason to learn Julia? JuMP is why I learned whatever I did of Julia and that was mostly because it allowed me to express some things better (not necessarily faster or in parallel). A good example would be this random problem that came out of an interview discussion. https://gist.github.com/t3rmin4t0r/44d8e09e17495d1c24908fc0f... I'm almost sure my python implementation is wrong, but I can't quite prove…

Using an optimization library seems like the completely wrong approach to solving this interview question. Was that intentional?

Re: Think Julia: How to Think Like a Computer Scientist

#28

Post by babahoyo »

zzleeper wrote: post #22
babahoyo wrote: post #14

Earlier quoted context omitted.

Julia is increasingly becoming better than R and Stata for data cleaning. Many of its metaprogramming tools beat `dplyr` in syntax and features. So if the data-cleaning to regression stack (which i would guess is different than scientific computing) is your thing, then i would recommend trying Julia out.

Early on (as a heavy Stata and Python user) I tried Julia and got quite discouraged by its messy treatment of missing values (and weights, etc). I've also tried R but also found lots of inconsistencies, so not enough reason to switch, besides when plotting nice graphs. But I would say Julia is increasingly getting there. Comparable packages are WAY easier to write in Julia than in Stata/Mata, while being faster, so a…

you were somehow satisfied with the way stata handles missing values???

    gen x = y if z > 4 // headaches abound
Julia's missing value support is great now and is only going to get better. You have to be more careful with how you use them, but you won't get anything like the output above in julia.

* For reference, stata uses +Inf as missing value, so any operation with "greater then" is going to assign missing values to something. And yes, there have research papers retracted due to this behavior.

Re: Think Julia: How to Think Like a Computer Scientist

#30

Post by dnautics »

fusiongyro wrote: post #25
mlthoughts2018 wrote: post #20

Earlier quoted context omitted.

Having spent years working with numpy and Cython, then switching to Scala for years as well, I much prefer dynamic typing. Strong type safety is mostly just a waste of time.

One thing that makes Julia cool is that adding the types improves performance, but you don't have to add them if you don't want to.

Adding types in julia usually does not improve performance for function definitions... it may improve performance for collections (like specifying the type that an array collects), but usually julia is pretty darn good at correctly inferring the collection type
Post reply on HN