81–90 of 106 posts
Earlier quoted context omitted.
Why should he care about other use-cases? It’s not his responsibility to make sure his package is as wide as possible before opensourcing.
The problem is not with the Wes' original decision but with the fact that it was never revisited even when pandas took off at much larger scope. Should had been fixed before 1.0 release.
It’s far easier to criticize than it is to submit a pull request.
I’m still hoping for a mixed-effects model implementation someday, like lme4 in R. The statsmodels implementation can only do predictions on fixed effects, which limits it greatly.
I’ve always wondered why mixed effect type models are not more popular in the ML world.
Earlier quoted context omitted.
Pandas was started by a quant working for AQR Capital, so it's not surprising if "Panda's choice is only good for high-frequency stock traders".
An illustrative example of how reasonable short-term and narrow-scope considerations can be really bad in long-term and/or at a larger scope.
Sometimes you just need a shovel, not a Bagger 288.
Excellent library for train_test_split. Jokes aside. This next to Numpy, Pandas Jupyter and Matplotlib + the DL libraries are the reason Python is the powerhouse it is for Data Science.
I'm with you on sklearn, the DL libraries and Numpy, but Pandas and Matplotlib are poor, poor relations of the tools available in the R ecosystem (dplyr/ggplot etc).
I like to stick to basic, widely used tools when possible so I'm biased against it versus just wrangling it out with matplotlib. But proplot does look compelling, like it was written for exactly my complaints.
Early on, pandas made some unfortunate design decisions that are still biting hard. For example, the choice of datetime (pandas.Timestamp) represented by a 64-bit int with a fixed nanosecond resolution. This choice gives dynamic range of +- 292 years around 1970-01-01 (the epoch). This range is too small to represent the works of William Shakespeare, never mind human history. Using pandas in these areas becomes a roy…
scikit-learn (next to numpy) is the one library I use in every single project at work. Every time I consider switching away from python I am faced with the fact that I'd lose access to this workhorse of a library. Of course it's not all sunshine and rainbows - I had my fair share of rummaging through its internals - but its API design is a de-facto standard for a reason. My only recurring gripe is that the serializat…
There is so much wrong with the api design of sklearn (how can one think "predict_proba" is a good function name?). I can understand this, since most of it was probably written by PhD students without the time and expertise to come up with a proper api; many of them without a CS background.[1] [1] https://www.reddit.com/r/haskell/comments/7brsuu/machine_lea...
Earlier quoted context omitted.
The problem is not with the Wes' original decision but with the fact that it was never revisited even when pandas took off at much larger scope. Should had been fixed before 1.0 release.
This belief is quite common in the Opensource space. It’s far easier to criticize than it is to submit a pull request.
Earlier quoted context omitted.
Wait how many companies are actually using R in the wild? As I understand it, R is born of academia, great for statistics/analysis but breaks down on data manipulation and isn't used in production/data engineering. Maybe my understanding is dated though?
R is everywhere, especially when you need to visualize stuff. It is primarily used in teams who are trying to get rid of SAS in my experience. You are right in the sense that R is typically not used end-to-end as far as I can tell, but already tries to start with a data connection to some sort of dump or datalake, or datawarehouse. Many people in my team use Python for modelling, but grab ggplot in whatever way to ma…
I primarily use Altair within Python. ggplot is ahead of Altair in some respects, but behind in others.
For example, here is a chart which can be made in Altair:
https://altair-viz.github.io/gallery/seattle_weather_interac...
Note:
- You can brush over the date range to filter the bar chart
- You can click on weather type to filter the scatter chart
- It can be embedded in any webpage with these interactive elements in tact. Since the chart is represented by json and rendered by javascript, the spec also embeds the data within the chart itself, and allows the user to therefore change the chart however they want
You can even build something like gapminder: https://vega.github.io/vega-lite/examples/interactive_global...
More examples here: https://altair-viz.github.io/gallery/
Earlier quoted context omitted.
maybe we are casualties of the vague definition of "data science," but in my experience numpy is too low-level for most of what I consider DS, and pandas/matplotlib are _much_ more central than sklearn or pytorch. Even if your definition only encompasses deep learning research, surely plotting is still indispensable? I'll also add my vote for the superiority of data.table and ggplot2 to any Python alternatives. the b…
Just curious. In which way is data.table superior to pandas? Really interested about it! From my personal experience pandas is just sometimes a bit slow.
If you already think pandas is slow I think you'll be surprised how much more strongly you feel after using data.table!