Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pandas vs. Julia – cheat sheet and comparison

datascientyst.com

Post reply on HN
Print view View original

101–110 of 138 posts

Re: Pandas vs. Julia – cheat sheet and comparison

#101

Post by _0w8t »

mrtranscendence wrote: post #84
Nanana909 wrote: post #76

Earlier quoted context omitted.

Any examples? I've found Julia far easier for simple things than python. Most modern problems are mathematical and nature and I think it's pretty objective that julia looks closer to the mathematics. ----Some tests of very simple tasks in both languages. At its most basic the obviouses becomes different: Let's try to get a very simple object: a 3,3 matrix of random booleans in both languages: Julia: A = rand(Bool,3,3…

It’s funny that people used to get on Python’s case for not being object oriented enough, and now we’ve come around to folks thinking Python should just throw a function for everything into the default namespace …

In Julia with multiple dispatch there is no problem to adding more things to the global namespace. But it does not work for Python, so it’s developers must be very conservative with global names.

Re: Pandas vs. Julia – cheat sheet and comparison

#102

Post by joeman1000 »

stoniejohnson wrote: post #64
joeman1000 wrote: post #56

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R: https://youtu.be/W3e8qMBypSE

Is that not also available in Pandas? https://pandas.pydata.org/docs/reference/api/pandas.DataFram...

Yes, but try using this and then try Julia's way. I tried this pandas implementation once and never touched it again.

Re: Pandas vs. Julia – cheat sheet and comparison

#103

Post by joeman1000 »

time_to_smile wrote: post #90
joeman1000 wrote: post #56

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R: https://youtu.be/W3e8qMBypSE

I'm still not entirely convinced that pipes aren't an anti-pattern. Absolutely an improvement over nested function calls: a(b(c(d))) vs d |> c |> b |> a but I'm not convinced pipes are better than more verbose code that explains each step: step1 = c(d) step2 = b(step1) result = a(step2) I've written a lot of tidy R and do understand the specific use cases where it really doesn't make sense to use the more verbose for…

I think having intermediate variables is sort of 'littering', and requires extra work in the naming which might not be necessary. Also, with pipes, you can just take out any intermediate step by commenting out a line or deleting it. You cannot do this with your method above without then going and rewriting many different arguments. I also like piping because you can quickly increment and build a solution - quicker than naming intermediate steps anyway.

Re: Pandas vs. Julia – cheat sheet and comparison

#104

Post by joeman1000 »

chaxor wrote: post #68
joeman1000 wrote: post #56

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R: https://youtu.be/W3e8qMBypSE

In pandas you can chain commands by wrapping the whole command in (). Personally IMO looks far 'cleaner' than all of the ugly %>% everywhere.

What do you mean by wrapping the command in ()? I haven't seen this before. Do you have a link to where they mention this in the docs?

Re: Pandas vs. Julia – cheat sheet and comparison

#105

Post by DNF2 »

kloch wrote: post #96
wnoise wrote: post #91

Earlier quoted context omitted.

> I'm not sure why this is the default behavior, Um. You said the answer earlier: > because the parser treats the literal as a double BigFloats aren't built in to the syntax of the language (and probably should't be, so you need to escape the parser somehow -- either pass a string to the constructor, or use the @big_str macro to get a non-standard string parsed into a BigFloat.

This makes perfect sense from the perspective of a language designer/computer scientist who is trying to keep their design clean and consistent. It makes no sense to an end user that expects an argument you pass to BigFloat to be treated as a BigFloat. As an end-user I would rather have a warning or even error than to have my argument silently treated as a double.

The argument isn't silently treated as a double, it is explicitly and loudly treated as a double, because it is a literal double.

And this is not an advantage to the designer exclusively, it is very much an advantage for the end user that the treatment is explicit, consistent and predictable, instead of 'magically' reinterpreting the meaning of literals based on guessing the intent of the user.

Basically, you seem to be saying that when passing x to BigFloat, x should not be treated as the value x, but as some nearby value that might be the one the caller intended (based on some rounding logic perhaps?) Or are you perhaps saying that

    x = 1e-300
    y = BigFloat(x)
should be different from

    y = BigFloat(1e-300)
? In other words, completely discarding referential transparency?

Re: Pandas vs. Julia – cheat sheet and comparison

#106

Post by joeman1000 »

ChrisRackauckas wrote: post #65
joeman1000 wrote: post #56

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R: https://youtu.be/W3e8qMBypSE

Julia has a pipe syntax (|>). But I think the bigger part here is more generally APIs built around it, which people are doing some things to port tidy syntax ( https://github.com/TidierOrg/Tidier.jl ).

This is very similar to DataFramesMeta:

https://github.com/JuliaData/DataFramesMeta.jl

Re: Pandas vs. Julia – cheat sheet and comparison

#107

Post by DNF2 »

patrick451 wrote: post #94
Nanana909 wrote: post #76

Earlier quoted context omitted.

Any examples? I've found Julia far easier for simple things than python. Most modern problems are mathematical and nature and I think it's pretty objective that julia looks closer to the mathematics. ----Some tests of very simple tasks in both languages. At its most basic the obviouses becomes different: Let's try to get a very simple object: a 3,3 matrix of random booleans in both languages: Julia: A = rand(Bool,3,3…

> Sorry, I mean my np.somenamespace.another.namespace.sparce head :) I really don't understand what you are trying to complain about. Namespaces are nice. Dumping everything into the global namespace sucks.

This is a problem for non-dispatch or singular dispatch languages, it's significantly different in the context of multiple dispatch. Namespaces are, well, not bad, but sometimes they are a solution to a problem that does not necessarily exist.

Re: Pandas vs. Julia – cheat sheet and comparison

#108

Post by gnramires »

DNF2 wrote: post #105
kloch wrote: post #96

Earlier quoted context omitted.

This makes perfect sense from the perspective of a language designer/computer scientist who is trying to keep their design clean and consistent. It makes no sense to an end user that expects an argument you pass to BigFloat to be treated as a BigFloat. As an end-user I would rather have a warning or even error than to have my argument silently treated as a double.

The argument isn't silently treated as a double, it is explicitly and loudly treated as a double, because it is a literal double. And this is not an advantage to the designer exclusively, it is very much an advantage for the end user that the treatment is explicit, consistent and predictable, instead of 'magically' reinterpreting the meaning of literals based on guessing the intent of the user. Basically, you seem to…

I think the argument here is that parsing of the string '1e-300' maybe should be context dependent. In this case, 1e-300 is being parsed as a double, and then forwarded to the function. Maybe it could be parsed as a bigfloat whenever it is an argument of a function expecting a bigfloat.

Re: Pandas vs. Julia – cheat sheet and comparison

#109

Post by affinepplan »

wdroz wrote: post #82
make3 wrote: post #78

Earlier quoted context omitted.

I wish Python catches up to Julia in performance. No sense rewriting a trillion lines of code for what is a really pleasant syntax & ecosystem already. But this is a language flamewar thing, probably not a constructive comment, sorry.

You can just use Polars[0] instead of Pandas and easily beat both Pandas and DataFrames.jl Pure Julia is faster than pure python, but there are non-pure python tools available in the python ecosystem for a ton of things. [0] -- https://www.pola.rs/

Polars definitely doesn't "easily" beat DF.jl on all tasks.

Yes, I agree, on average polars is a bit faster for many of the simple workflows, but I certainly don't think that's unconditionally true. It's especially less true when you might want to do something out of the ordinary with your series --- in Julia it's trivial to just extract that as a vector and loop over it (fast!). In polars, one would have to make sure their function can be appropriately vectorized.

Re: Pandas vs. Julia – cheat sheet and comparison

#110

Post by jakobnissen »

gnramires wrote: post #108
DNF2 wrote: post #105

Earlier quoted context omitted.

The argument isn't silently treated as a double, it is explicitly and loudly treated as a double, because it is a literal double. And this is not an advantage to the designer exclusively, it is very much an advantage for the end user that the treatment is explicit, consistent and predictable, instead of 'magically' reinterpreting the meaning of literals based on guessing the intent of the user. Basically, you seem to…

I think the argument here is that parsing of the string '1e-300' maybe should be context dependent. In this case, 1e-300 is being parsed as a double, and then forwarded to the function. Maybe it could be parsed as a bigfloat whenever it is an argument of a function expecting a bigfloat.

Julia does this for exponentiation, i.e. a literal exponent is parsef differently that exponentiation by a variable.

I think it was a mistake. Invariably, a new user discovers the discrepancy and is thoroughly confused. Let's not repeat the same mistake with big nums.

Post reply on HN