Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pandas vs. Julia – cheat sheet and comparison

datascientyst.com

Post reply on HN
Print view View original

51–60 of 138 posts

Re: Pandas vs. Julia – cheat sheet and comparison

#51

Post by marginalia_nu »

EuAndreh wrote: post #41
marginalia_nu wrote: post #6

Earlier quoted context omitted.

You do a lot of software development on mobile?

We should nurture more accessibility, in this case, mobile compatibility. For instance, consider someone who has limited access to desktop computers and have to go by with a mobile device. These individuals do exist, and their access is as legitimate as any other.

Mobile compatible websites are strictly worse though. It's why almost all desktop websites are just a hideous jumble of boxes these days.

It's virtually impossible to make a website that is well designed on both desktop and mobile. As long as the affordances of mouse+keyboard and touchscreen are as different as they are, one of the user groups needs will suffer a detrimental compromise.

Re: Pandas vs. Julia – cheat sheet and comparison

#52

Post by freilanzer »

avnigo wrote: post #42
agacera wrote: post #31

Earlier quoted context omitted.

Same here. But have you tried duckdb? You can do sql in the pandas dfs and it is fast af. https://duckdb.org/2021/05/14/sql-on-pandas.html

mydf = pd.DataFrame({'a' : [1, 2, 3]}) print(duckdb.query("SELECT SUM(a) FROM mydf").to_df()) I can see the appeal, but if you're working in Python, something doesn't sit right with me when having to write out variable names as strings. E.g., if I want to refactor the code, my LSP or parser won't pick up those references. > The SQL table name mydf is interpreted as the local Python variable mydf [...] Not only is thi…

https://duckdb.org/docs/guides/python/ibis.html

Re: Pandas vs. Julia – cheat sheet and comparison

#54

Post by martinsmit »

cookieperson wrote: post #33
SilverBirch wrote: post #11

Yeah this is basically why I keep trying and bouncing off Julia. I understand the real performance reasons why you'd choose to use Julia but the syntax is the perfect distance from python to make it extremely difficult to me. It's just close enough to get constantly confused. So if I really wanted to do much work in it I'd have swear off python - and I can't do that because for trivial stuff python is more convenient…

Pretty sure dataframes jl isn't the fastest dataframes library out there. Think it's Polars, which has bindings in Rust and python. If I remember correctly the runner up is data.table. Similarly SQL/SQLite can often beat all of these So switching to Julia for speed in this context may not even make sense anyways...

I agree with your conclusion but want to add that switching from Julia may not make sense either.

According to these benchmarks: https://h2oai.github.io/db-benchmark/, DF.jl is the fastest library for some things, data.table for others, polars for others. Which is fastest depends on the query and whether it takes advantage of the features/properties of each.

For what it's worth, data.table is my favourite to use and I believe it has the nicest ergonomics of the three I spoke about.

Re: Pandas vs. Julia – cheat sheet and comparison

#56

Post by joeman1000 »

The thing that keeps me coming back to Julia is the ability to pipe (or whatever you want to call it). It makes DataFrame operations a lot cleaner since I don't need to modify in place or create new DFs at intermediate steps in a process. Here's a video showing this sort of workflow in R:

https://youtu.be/W3e8qMBypSE

Re: Pandas vs. Julia – cheat sheet and comparison

#57

Post by joeman1000 »

rgavuliak wrote: post #37

The cheatsheet goes wrong already for the first example of declaring a df: - you could do a range in python (range(11, 14)) - columns are called col_1 & col_2 vs a & b (both sets are horrible names) - pandas defines index of 0, 1, 3, while Julia would most likely have 0, 1, 2?

What do you mean? Julia is 1-indexed.

Re: Pandas vs. Julia – cheat sheet and comparison

#58

Post by affinepplan »

cookieperson wrote: post #34
freilanzer wrote: post #26

Earlier quoted context omitted.

I have done both complex and trivial stuff in both languages and Julia isn't more inconvenient for trivial things.

Just make sure you find the appropriate documentation because the package changes it's syntax an awful lot over the past four years or so and there are lots of tutorials, videos, and blogs that don't apply anymore. Similarly make sure you research the ecosystem because everything in Julia is very fragmented, IE pandas.loadcsv will require two or more packages in it's Julia equivalent.

I notice you coming into every single thread about Julia to criticize the language and the community. Do you have a vendetta or something?

Re: Pandas vs. Julia – cheat sheet and comparison

#59

Post by affinepplan »

adammarples wrote: post #24

It's strange, I would think that Julia's multiple dispatch would make something more like this desirable ``` df = DataFrame(CSV(File("name.csv"))) data = JSON(File("name.json")) ``` instead of the usual hodge podge of methods ``` df = CSV.read("file.csv", DataFrame) data = JSON.parsefile("file.json") ```

Actually, that works too :) I think `DataFrame(CSV.File("name.csv"))` is what you're looking for

Re: Pandas vs. Julia – cheat sheet and comparison

#60

Post by Nilshg »

cookieperson wrote: post #34
freilanzer wrote: post #26

Earlier quoted context omitted.

I have done both complex and trivial stuff in both languages and Julia isn't more inconvenient for trivial things.

Just make sure you find the appropriate documentation because the package changes it's syntax an awful lot over the past four years or so and there are lots of tutorials, videos, and blogs that don't apply anymore. Similarly make sure you research the ecosystem because everything in Julia is very fragmented, IE pandas.loadcsv will require two or more packages in it's Julia equivalent.

To be clear on this: DataFrames, like most of the Julia ecosystem, follows SemVer. DataFrames 1.0 was released over two years ago (March 2021), and the API has been stable ever since.

Furthermore, Bogumil Kaminski, one of the main developers behind DataFrames, makes sure that the DataFrames tutorials he has created here (https://github.com/bkamins/Julia-DataFrames-Tutorial) are updated on every new release.

Post reply on HN