Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Directed Pandas Crash Course

kellyfoulk.herokuapp.com

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Self-Directed Pandas Crash Course

#31

Post by efxhoy »

mft_ wrote: post #28
chaps wrote: post #7

Earlier quoted context omitted.

100% my experience whenever I work with it, and I've been working with it on and off for about five years. I get its appeal -- it fits that fuzzy place where a database is too heavy, but vanilla python is cumbersome... but damn, is it tough to work with. With almost no fail, I always seem to just scrap the pandas code in favor of vanilla python, either from usability issues, or from yet another a library's false prom…

Funny that the example you give is one of the (many) frustrations with Pandas. It’s ‘read_ ’ to read something in, but ‘to_ ’ to write it out? In which world is this intuitive? Surely it should be read/write or even from/to?

It's `pandas.read_ -> DataFrame` and `DataFrame.to_`. So pandas reads a data format and gives you a dataframe. The `to_` are instane methods of the DataFrame where you write a dataframe to a format. To me that makes sense.

Re: Self-Directed Pandas Crash Course

#32

Post by trumpeta »

dwohnitmok wrote: post #8
systemvoltage wrote: post #3

Pandas is basically impossible for me to use without dozens of Google searches after being familiar with it for over 7 years. Ofcourse, I don't use it daily but its one of those pieces of software that has a very non-intuitive API. Does any one find it difficult to use it or its just me? In particular, I find this answer infuriating [1]. I've come across it so many times. Look, I have a CSV with 200 rows and I need t…

I understand where you're coming from (yes yes there's a theoretically "good" way of doing this, but come on, why can't I just do the simplest thing), but I also sympathize with the spirit of that SO answer (although I agree that its presentation is wanting). Pandas is really a DSL unto itself and is heavily influenced by R, where the same dynamic happens. Programmers coming from a background where procedural control…

You’re making a great point there about procedural mindset being applied to array programming. But the thing is, I feel like array based programming should lend itself naturally to functional approaches. And Pandas does do this to an extent.

My problem is that this is super inconsistent. Some things are done as a method call on an object, others by passing the object to a pandas function and others yet by passing a function to a method on an object. This is the major source of frustration for me.

Maybe there is some logic to that, but I haven’t found it yet and I think that is a sign of bad API design. Its like PHP to me. All nice and documented but useless without Googling everything

Re: Self-Directed Pandas Crash Course

#33

Post by Bostonian »

smabie wrote: post #15
dwohnitmok wrote: post #8

Earlier quoted context omitted.

I understand where you're coming from (yes yes there's a theoretically "good" way of doing this, but come on, why can't I just do the simplest thing), but I also sympathize with the spirit of that SO answer (although I agree that its presentation is wanting). Pandas is really a DSL unto itself and is heavily influenced by R, where the same dynamic happens. Programmers coming from a background where procedural control…

I actually don't like Pandas but I think it's pretty obvious that the array language style is an incredibly powerful way to manipulate blocks of multiple dimensional data. People here are complaining about not being able to use for loops, but why would you want to use them in the first place? Like let's say you want to add two vectors together. Looping over each index and adding them and assigning them to a third vec…

Fortran has had array operations since the 1990 standard, but I believe that programmers who are familiar with that functionality still often use loops involving array elements. I do. Not all calculations conveniently map to array operations, and sometimes you can exit the loop if a condition is met. It's a benefit of a compiled, statically typed language to use loops without suffering a speed penalty.

Re: Self-Directed Pandas Crash Course

#34

Post by ivansavz »

Khuyen Tran has a nice collection of bite-sized tricks and tips for learning Pandas on her blog: https://mathdatasimplified.com/page/1/?s=pandas https://mathdatasimplified.com/page/2/?s=pandas https://mathdatasimplified.com/page/3/?s=pandas

It's not like I'll remember all the syntax, but good to know what tools/tricks exist.

Re: Self-Directed Pandas Crash Course

#35

Post by PlugTunin »

asicsp wrote: post #9

Here's another recent resource: https://www.gormanalysis.com/blog/python-pandas-for-your-gra...

Subscribed. Thanks for putting together this quality content. I was checking out your merge dataframes video the other day, that I was led to from a Stack Overflow post. I'm not a novice w/Pandas but I find that things stick better after repeated exposure, from different sources at that. The various ways of using merge/join/concat can be tricky to keep straight at first. I will be sure to dig deeper into your content in the near future, including the series of videos on Numpy.

Re: Self-Directed Pandas Crash Course

#36

Post by asicsp »

PlugTunin wrote: post #35
asicsp wrote: post #9

Here's another recent resource: https://www.gormanalysis.com/blog/python-pandas-for-your-gra...

Subscribed. Thanks for putting together this quality content. I was checking out your merge dataframes video the other day, that I was led to from a Stack Overflow post. I'm not a novice w/Pandas but I find that things stick better after repeated exposure, from different sources at that. The various ways of using merge/join/concat can be tricky to keep straight at first. I will be sure to dig deeper into your content…

Not mine, I'd have mentioned if it was my resource.

I saw it on: https://www.reddit.com/r/Python/comments/lain0r/hey_reddit_h...

Re: Self-Directed Pandas Crash Course

#37

Post by greenie_beans »

systemvoltage wrote: post #3

Pandas is basically impossible for me to use without dozens of Google searches after being familiar with it for over 7 years. Ofcourse, I don't use it daily but its one of those pieces of software that has a very non-intuitive API. Does any one find it difficult to use it or its just me? In particular, I find this answer infuriating [1]. I've come across it so many times. Look, I have a CSV with 200 rows and I need t…

This is reassuring. Anytime I use pandas I’m like wtf? (First encountered it like a year ago)

Re: Self-Directed Pandas Crash Course

#38

Post by mft_ »

efxhoy wrote: post #31
mft_ wrote: post #28

Earlier quoted context omitted.

Funny that the example you give is one of the (many) frustrations with Pandas. It’s ‘read_ ’ to read something in, but ‘to_ ’ to write it out? In which world is this intuitive? Surely it should be read/write or even from/to?

It's `pandas.read_ -> DataFrame` and `DataFrame.to_ `. So pandas reads a data format and gives you a dataframe. The `to_ ` are instane methods of the DataFrame where you write a dataframe to a format. To me that makes sense.

Sure, you can create a sort-of logic to fit the existing naming, but it's still something you have to learn.

Imagine you were using Pandas for the first time, working through a tutorial, and you've just learned that pd.read_csv reads a csv into a dataframe.

Intuitively, what would you expect the corresponding output function to be called? I'd hazard that a vast majority of people would guess at some version of write_csv, and would experiment with either pd.write_csv, or df.write_csv.

Re: Self-Directed Pandas Crash Course

#39

Post by edmundsauto »

glenngillen wrote: post #27
edmundsauto wrote: post #24

Earlier quoted context omitted.

For folks doing this everyday, absolutely. For anyone just learning, it's Yet Another Thing to learn. Sure, the complete programmer will be able to think in better abstractions, but most people fail to reach that level. In part because of the complexity of early stage learning. It's a balance, and I think pandas is on the more complex side of things.

If you don’t want the conceptual model and features pandas provides, is there a reason you don’t just use regular Python without pandas?

Pandas has really great ingesting / exporting helpers, is easy to chart, and lots of tutorials reach for it first.

Those might be more excuses than reasons, but it’s my experience.

Re: Self-Directed Pandas Crash Course

#40

Post by gk1 »

yardshop wrote: post #6
nolroz wrote: post #2

I like the outline you have here for a crash course on Pandas. I've been tinkering with it myself off and on for a while but have been wanting to to really dig in lately for the same reasons you mention. Just a couple of nitpicks: 1. The style of your website makes it pretty much impossible to tell if a bit of text is linked anywhere. I only figured it out after clicking on the word "here" in the first paragraph and…

He links it in the first sentence under Resources. As you mention, it's a "here" link with no styling. One shouldn't have to, but FWIW the HTML of his page is very clean so you can see all of the links by viewing the source.

*She
Post reply on HN