Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia Macros for Beginners

jkrumbiegel.com

Post reply on HN
Print view View original

21–30 of 36 posts

Re: Julia Macros for Beginners

#21

Post by cbkeller »

medo-bear wrote: post #9
sischoel wrote: post #4

Earlier quoted context omitted.

One could argue that Julia is just Lisp with a syntax that appeals more to popular taste. There is also a secret option to get into a lisp repl in Julia "julia --lisp".

yeah it doesn't take much eye squinting to see lisp in julia. given these similarities i wonder if julia users could start to appreciate the s-expression syntax. i come from matlab then python background and i have come to really enjoy the s-expression syntax. modern IDE tools have made s-expression code as readable as pythonic pseudo-code-like syntax while affording the programmer unrivaled editing power

As it happens, in addition to the femptolisp in the Julia parser, there is actually a secret s-expression syntax for Julia itself. There's no built in REPL mode for it, but you can hack one in about a dozen lines: https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc76...

Re: Julia Macros for Beginners

#22

Post by Tyr42 »

pdeffebach wrote: post #17
coop_solution wrote: post #15

Earlier quoted context omitted.

What does dplyr syntax look like?

Let's say you have a data frame df = tibble(a = c(1, 2)) and you want to use a dplyr verb to modify it mutate(df, b = a + 1) the `a` in the above expression refers to the column in `df`, but this means it's hard to reference a variable in the outer scope named `a`. Furthermore, if you have a string referring to the column name `"a"`, you can't simply write mutate(df, b = a_var + 1) Contrast this with DataFramesMeta.j…

It's even better when you have the "." variable which get populated.

But in general yeah, R plays pretty fast and loose with scopes, and lets you capture expressions as arguments and execute them in a different scope from the outside one

Re: Julia Macros for Beginners

#23

Post by medo-bear »

cbkeller wrote: post #21
medo-bear wrote: post #9

Earlier quoted context omitted.

yeah it doesn't take much eye squinting to see lisp in julia. given these similarities i wonder if julia users could start to appreciate the s-expression syntax. i come from matlab then python background and i have come to really enjoy the s-expression syntax. modern IDE tools have made s-expression code as readable as pythonic pseudo-code-like syntax while affording the programmer unrivaled editing power

As it happens, in addition to the femptolisp in the Julia parser, there is actually a secret s-expression syntax for Julia itself. There's no built in REPL mode for it, but you can hack one in about a dozen lines: https://gist.github.com/brenhinkeller/44051118c2f9d18b26dc76...

nice! one annoying nit pick for me though is using commas as data separators. when you need to input data by hand into a multi dim array this can get annoying very quickly

Re: Julia Macros for Beginners

#24

Post by dan-robertson »

The frustration of Julia macros for me was never knowing what AST would be produced for a given expression. This is a bit more manageable if you have a typed AST (e.g. OCaml but ppxes have other issues) or an obvious one (e.g. lisp). I like the way rust handles it where the macros operate on a tree of non-delimiter leaves and [delimiter, subtree list, delimiter] nodes which can allow for figuring out what the input to a macro will be more easily and for more varied macro syntax. Other languages that want a full AST before macros force the macro input to be a bit more AST-like, e.g. the Julia parser picks operator precedence and OCaml won’t let you use _ as an identifier.

Maybe it is better now but when I looked at macros ~5 years ago some language update changed the ast produced by the parser and I basically gave up.

I like that Julia offers some macro-like techniques that replace a lot of the cases where one might use a macro for performance reasons.

Re: Julia Macros for Beginners

#25

Post by srcreigh »

Kalanos wrote: post #14

never understood why this was better/ how it was different from regular functions. just seems like bugs/vulns waiting to happen

Some macro systems can create variables in a loop for you. You could make this macro,

     (define-all i 5 0)
     ;; creates i1 i2 i3 i4 i5 initialized to 0
That's somewhat impossible with functions. The closest you get is either an array/dict with only runtime error checking, or an external codegen program.

I wrote a post [0] about how to do this in Racket. The macro generates ORM code based given a SQLite DB. Aka the compiler queries SQLite and generates table-column functions automatically.

More potential benefits are: Better static error messages (can implement a type system using macros, example here[1]), and controlling execution order (can add lazy computation semantics).

[0]: http://tech.perpetua.io/2022/01/generating-sqlite-bindings-w...

[1]: https://gist.github.com/srcreigh/f341b2adaa0fe37c241fdf15f37...

Re: Julia Macros for Beginners

#26

Post by ryanmonroe »

pdeffebach wrote: post #17
coop_solution wrote: post #15

Earlier quoted context omitted.

What does dplyr syntax look like?

Let's say you have a data frame df = tibble(a = c(1, 2)) and you want to use a dplyr verb to modify it mutate(df, b = a + 1) the `a` in the above expression refers to the column in `df`, but this means it's hard to reference a variable in the outer scope named `a`. Furthermore, if you have a string referring to the column name `"a"`, you can't simply write mutate(df, b = a_var + 1) Contrast this with DataFramesMeta.j…

To reference variables in the outer scope, you would do

    mutate(df, b = .env$a + 1)
And if you have a string (contained in a_var) which identifies a variable you can do

    mutate(df, b = .data[[a_var]] + 1)
You could argue these feel clumsy, but I wouldn’t say it’s “hard” to do either of these things with dplyr.

Re: Julia Macros for Beginners

#27

Post by dunefox »

dan-robertson wrote: post #24

The frustration of Julia macros for me was never knowing what AST would be produced for a given expression. This is a bit more manageable if you have a typed AST (e.g. OCaml but ppxes have other issues) or an obvious one (e.g. lisp). I like the way rust handles it where the macros operate on a tree of non-delimiter leaves and [delimiter, subtree list, delimiter] nodes which can allow for figuring out what the input t…

> ~5 years ago

Was the language even stable then?

Re: Julia Macros for Beginners

#28

Post by krumbie »

ryanmonroe wrote: post #26
pdeffebach wrote: post #17

Earlier quoted context omitted.

Let's say you have a data frame df = tibble(a = c(1, 2)) and you want to use a dplyr verb to modify it mutate(df, b = a + 1) the `a` in the above expression refers to the column in `df`, but this means it's hard to reference a variable in the outer scope named `a`. Furthermore, if you have a string referring to the column name `"a"`, you can't simply write mutate(df, b = a_var + 1) Contrast this with DataFramesMeta.j…

To reference variables in the outer scope, you would do mutate(df, b = .env$a + 1) And if you have a string (contained in a_var) which identifies a variable you can do mutate(df, b = .data[[a_var]] + 1) You could argue these feel clumsy, but I wouldn’t say it’s “hard” to do either of these things with dplyr.

I don't think it's just about whether it's hard to do, your syntax example looks short enough and one can memorize these two patterns relatively quickly.

However, both patterns are another special case how identifiers are resolved in the expression. Aren't `.env` and `.data` both valid variable and column names? So what happens if I have a column named `.data`?

Another example, which is the reason why we chose the `:column` style to refer to columns in `DataFramesMeta.jl` and `DataFrameMacros.jl`:

What happens if you have the expression `mutate(df, b = log(a))`. Both `log` and `a` are symbols, but `log` is not treated as a column. Maybe that's because it's used in a function-like fashion? Maybe because R looks at the value of `log` and `a` in their scope and sees that `log` is a function an `a` isn't?

In Julia DataFrames, it's totally valid to have a column that stores different functions. With the dplyr like syntax rules it would not be possible to express a function call with a function stored in a column, if the pattern really is that function syntax means a symbol is not looked up in the dataframe anymore.

In Julia DataFrameMacros.jl for example, if you had a column named `:func` you could do `@transform(df, :b = :func(:a))` and it would be clear that `:func` resolves to a column.

This particular example might seem like a niche problem, but it's just one of these tradeoffs that you have to make when overloading syntax with a different meaning. I personally like it if there's a small rule set which is then consistently applied. I'd argue that's not always the case with dplyr.

Re: Julia Macros for Beginners

#29

Post by adgjlsfhk1 »

dunefox wrote: post #27
dan-robertson wrote: post #24

The frustration of Julia macros for me was never knowing what AST would be produced for a given expression. This is a bit more manageable if you have a typed AST (e.g. OCaml but ppxes have other issues) or an obvious one (e.g. lisp). I like the way rust handles it where the macros operate on a tree of non-delimiter leaves and [delimiter, subtree list, delimiter] nodes which can allow for figuring out what the input t…

> ~5 years ago Was the language even stable then?

5 years ago was right before 1.0, so a ton of stuff was broken since it was the last chance.

Re: Julia Macros for Beginners

#30

Post by pdeffebach »

ryanmonroe wrote: post #26
pdeffebach wrote: post #17

Earlier quoted context omitted.

Let's say you have a data frame df = tibble(a = c(1, 2)) and you want to use a dplyr verb to modify it mutate(df, b = a + 1) the `a` in the above expression refers to the column in `df`, but this means it's hard to reference a variable in the outer scope named `a`. Furthermore, if you have a string referring to the column name `"a"`, you can't simply write mutate(df, b = a_var + 1) Contrast this with DataFramesMeta.j…

To reference variables in the outer scope, you would do mutate(df, b = .env$a + 1) And if you have a string (contained in a_var) which identifies a variable you can do mutate(df, b = .data[[a_var]] + 1) You could argue these feel clumsy, but I wouldn’t say it’s “hard” to do either of these things with dplyr.

It would be interesting to profile the 2nd version though. Assuming the non-standard evaluation has performance benefits (which they do in DataFramesMeta.jl), are you eliminating those benefits when you use

    .data[[a_var]]

?
Post reply on HN