Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pandas vs. Julia – cheat sheet and comparison

datascientyst.com

Post reply on HN
Print view View original

91–100 of 138 posts

Re: Pandas vs. Julia – cheat sheet and comparison

#91

Post by wnoise »

kloch wrote: post #86

I've just started getting into Julia for one of it's best use cases: It's super easy to do arbitrary precision math. But you have to be very careful when using string literals with BigInt or BigFloat: julia> setprecision(1024) julia> a=BigFloat(1.0E-300) 1.0000000000000000250590918352087596856961468077037052499253423199004660431840514846763028121819501008949623062702782541489103114649988041308122460916061901827194266…

> I'm not sure why this is the default behavior,

Um. You said the answer earlier:

> because the parser treats the literal as a double

BigFloats aren't built in to the syntax of the language (and probably should't be, so you need to escape the parser somehow -- either pass a string to the constructor, or use the @big_str macro to get a non-standard string parsed into a BigFloat.

Re: Pandas vs. Julia – cheat sheet and comparison

#92

Post by patrick451 »

affinepplan wrote: post #58
cookieperson wrote: post #34

Earlier quoted context omitted.

Just make sure you find the appropriate documentation because the package changes it's syntax an awful lot over the past four years or so and there are lots of tutorials, videos, and blogs that don't apply anymore. Similarly make sure you research the ecosystem because everything in Julia is very fragmented, IE pandas.loadcsv will require two or more packages in it's Julia equivalent.

I notice you coming into every single thread about Julia to criticize the language and the community. Do you have a vendetta or something?

The Rust Evangalism Strike force normalized the practice. This is what we do now.

Re: Pandas vs. Julia – cheat sheet and comparison

#93

Post by hpcjoe »

cookieperson wrote: post #34
freilanzer wrote: post #26

Earlier quoted context omitted.

I have done both complex and trivial stuff in both languages and Julia isn't more inconvenient for trivial things.

Just make sure you find the appropriate documentation because the package changes it's syntax an awful lot over the past four years or so and there are lots of tutorials, videos, and blogs that don't apply anymore. Similarly make sure you research the ecosystem because everything in Julia is very fragmented, IE pandas.loadcsv will require two or more packages in it's Julia equivalent.

> Julia is very fragmented

No, it isn't. I'm using Pandas, DF.jl, and even polars at work. DF.jl is by far the best/easiest/quickest to use, as its syntax is consistent. Polars is a bit more annoying as its syntax is further along the learning curve that I have gotten yet.

Pandas ... what to say about a library that will happily return a pd.Series in one moment, and a pd.DataFrame in another, for the same function call. This means you need extra code like

if type(ret_thing) = pd.Series: # then do something to coax it back to a df.

lest your actual code break.

This is of course the same language that has API differences that make no sense in, say, re.match vs re.findall vs re.search. I've been burned by all of those.

So, look, we get you hate Julia. That's fine. Go live your python life to its best. But really, stop with the misinformation/FUD. This speaks volumes about you, and tends to make the case precisely the opposite of what you think.

And yes, I use Python, Julia, C++, and many other languages in the $day_job.

Re: Pandas vs. Julia – cheat sheet and comparison

#94

Post by patrick451 »

Nanana909 wrote: post #76
freilanzer wrote: post #26

Earlier quoted context omitted.

I have done both complex and trivial stuff in both languages and Julia isn't more inconvenient for trivial things.

Any examples? I've found Julia far easier for simple things than python. Most modern problems are mathematical and nature and I think it's pretty objective that julia looks closer to the mathematics. ----Some tests of very simple tasks in both languages. At its most basic the obviouses becomes different: Let's try to get a very simple object: a 3,3 matrix of random booleans in both languages: Julia: A = rand(Bool,3,3…

> Sorry, I mean my np.somenamespace.another.namespace.sparce head :)

I really don't understand what you are trying to complain about. Namespaces are nice. Dumping everything into the global namespace sucks.

Re: Pandas vs. Julia – cheat sheet and comparison

#95

Post by hpcjoe »

affinepplan wrote: post #58
cookieperson wrote: post #34

Earlier quoted context omitted.

Just make sure you find the appropriate documentation because the package changes it's syntax an awful lot over the past four years or so and there are lots of tutorials, videos, and blogs that don't apply anymore. Similarly make sure you research the ecosystem because everything in Julia is very fragmented, IE pandas.loadcsv will require two or more packages in it's Julia equivalent.

I notice you coming into every single thread about Julia to criticize the language and the community. Do you have a vendetta or something?

Back in my more perl-ish days, I recall pythonistas doing this. Sad to see little has changed.

Re: Pandas vs. Julia – cheat sheet and comparison

#96

Post by kloch »

wnoise wrote: post #91
kloch wrote: post #86

I've just started getting into Julia for one of it's best use cases: It's super easy to do arbitrary precision math. But you have to be very careful when using string literals with BigInt or BigFloat: julia> setprecision(1024) julia> a=BigFloat(1.0E-300) 1.0000000000000000250590918352087596856961468077037052499253423199004660431840514846763028121819501008949623062702782541489103114649988041308122460916061901827194266…

> I'm not sure why this is the default behavior, Um. You said the answer earlier: > because the parser treats the literal as a double BigFloats aren't built in to the syntax of the language (and probably should't be, so you need to escape the parser somehow -- either pass a string to the constructor, or use the @big_str macro to get a non-standard string parsed into a BigFloat.

This makes perfect sense from the perspective of a language designer/computer scientist who is trying to keep their design clean and consistent.

It makes no sense to an end user that expects an argument you pass to BigFloat to be treated as a BigFloat. As an end-user I would rather have a warning or even error than to have my argument silently treated as a double.

Re: Pandas vs. Julia – cheat sheet and comparison

#97

Post by civilized »

civilized wrote: post #73
chaxor wrote: post #68

Earlier quoted context omitted.

In pandas you can chain commands by wrapping the whole command in (). Personally IMO looks far 'cleaner' than all of the ugly %>% everywhere.

The flip side is that in pandas, chaining is less uniform because it is based on methods. In R you can pipe a data frame into any function from any package or one you just wrote, so you use %>% for any piping that happens. In pandas, you have special pandas methods that don't need the pipe, but to pipe with any other function, you have to write .pipe. The comparison is not really between %>% and ., it's between "you…

The sad thing about the conventional object-oriented programming paradigm is how it put the really cool syntactic idea of piping/chaining in the straitjacket of classes and objects.

The ability to pipe shouldn't be tied to whether a function is a method of a class.

Re: Pandas vs. Julia – cheat sheet and comparison

#98

Post by sundarurfriend »

kloch wrote: post #96
wnoise wrote: post #91

Earlier quoted context omitted.

> I'm not sure why this is the default behavior, Um. You said the answer earlier: > because the parser treats the literal as a double BigFloats aren't built in to the syntax of the language (and probably should't be, so you need to escape the parser somehow -- either pass a string to the constructor, or use the @big_str macro to get a non-standard string parsed into a BigFloat.

This makes perfect sense from the perspective of a language designer/computer scientist who is trying to keep their design clean and consistent. It makes no sense to an end user that expects an argument you pass to BigFloat to be treated as a BigFloat. As an end-user I would rather have a warning or even error than to have my argument silently treated as a double.

It's a tricky case because they do provide the `big""` macros for literals, and mention in BigFloat's docs that:

      BigFloat(x::AbstractString) is identical to parse. This is provided for convenience since decimal literals are converted to
      Float64 when parsed, so BigFloat(2.1) may not yield what you expect.

      ...
      Examples
      ≡≡≡≡≡≡≡≡≡≡

      julia> BigFloat(2.1) # 2.1 here is a Float64
      2.100000000000000088817841970012523233890533447265625
However, saying RTFM is not a solution, especially for not-too-frequent parts of the language like BigFloats. It's still a trap many people are going to fall for.

The solution here is a good linter though, not adding more work to the already overstressed compiler. It comes back to the issue of Julia needing more mature, easy-to-work-with tooling, that could say "hey, this is technically allowed, but you probably didn't mean this".

Re: Pandas vs. Julia – cheat sheet and comparison

#99

Post by sundarurfriend »

tombert wrote: post #19
SilverBirch wrote: post #11

Yeah this is basically why I keep trying and bouncing off Julia. I understand the real performance reasons why you'd choose to use Julia but the syntax is the perfect distance from python to make it extremely difficult to me. It's just close enough to get constantly confused. So if I really wanted to do much work in it I'd have swear off python - and I can't do that because for trivial stuff python is more convenient…

I have not done anything even remotely significant in Julia, but the little I played with didn't seem to indicate to me that it would be bad for trivial stuff...what trivial stuff is hard in Julia but easy in Python?

I think they just mean that with trivial projects, it's not worth trying them in a new language since the performance benefits are probably going to be minimal, and wouldn't really show off Julia's strengths.

Re: Pandas vs. Julia – cheat sheet and comparison

#100

Post by _0w8t »

time_to_smile wrote: post #90
joeman1000 wrote: post #56

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R: https://youtu.be/W3e8qMBypSE

I'm still not entirely convinced that pipes aren't an anti-pattern. Absolutely an improvement over nested function calls: a(b(c(d))) vs d |> c |> b |> a but I'm not convinced pipes are better than more verbose code that explains each step: step1 = c(d) step2 = b(step1) result = a(step2) I've written a lot of tidy R and do understand the specific use cases where it really doesn't make sense to use the more verbose for…

Naming intermediate steps require some non-trivial efforts. It can even distract from the main task of getting the results.

In programming the code will be read multiple times and good names will help the future readers. But in data science the calculation will be most likely will not be reused. So efforts to name things will be waste of time.

Post reply on HN