Live data from Hacker News

No, shut up. What statistical programming languages can learn from Dropbox.

erehweb.wordpress.com

41–50 of 58 posts

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#41
> People don’t use that crap. > But they do want pivot tables, ...

That's what the cookbook recipe provides, a function called summary() that makes a pivot table. Problem solved :-)

> I should be clear that my complaint is with > Python rather than the code as such.

There are plenty of ways to write the summary() function with plain, straight-forward Python code that doesn't use generators, itertools, or any other advanced feature.

So, why does the recipe author use itertools? It is because they provide a way to get C speed without having to write extension modules. Had the author used map() instead of a generator expression, the inner-loop would run entirely at C speed (with no trips around the Python eval-loop):

  for pivot_value, row in groupby(data, key):
      yield k, sum(map(value, group))
I think it's wonderful that a two-line helper function is all it takes to implement pivot tables efficiently.

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#42
post #32

To be honest I find the whole repeated "no, shut up" thing to be a bit crass and it makes me unsympathetic if anything. I hope this doesn't become a catchphrase in blogs.

I thought it was perfect in the original dropbox quora post, but I agree that it doesn't quite fit here, and there is certainly a danger of it becoming a meme.

[deleted]

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#43

Earlier quoted context omitted.

There are GUI statistics apps for people who just want the common case, Dropbox-style: packages like Weka for data mining / predictive statistics, SPSS for descriptive statistics, and a dozen other such things. The statisticians who choose to use a programming language like R or Python typically do it because they actually do want a programming language. I mean, that's why Bell Labs statisticians invented S (R's pred…

Right. I wasn't saying that there didn't exist such packages, of course there are. I was pointing out that the reason a programming language looks good to a programmer and not a statistician is due domain expertise. And of course the common trap programmers fall into is assuming the domain is programming. And don't lump R in with Python. And good statistician would have your neck. You mention S, but again S doesn't l…

I only see him "lumping R in with Python" in that they're both full-blown programming languages and TFAA apparently hates them both because they're programming languages.

_delirium is merely pointing out that there are push-button packages for statistics, and that statisticians using programming languages (be they statistics-oriented or not) usually do so because they want to or because they need to (as the push-button stuff is not sufficient for their needs, for instance)

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#45
post #13

If 90% of usage boils down to a small number of rigid patterns, then there is a simple solution: a handful of convenience functions. Often these functions are missing, because the demand for convenience functions is obscured by the fact that every experienced user defined them for himself years ago. That forces newbies to suffer through the unnecessary task of understanding the fully generalized API before they can a…

Exactly. It's trivial to write a prettied-up interface to those Python functions that would make as much sense (?) as PROC MEANS. Good luck trying extend SAS to do anything the designers didn't implement as a procedure, though. Having had to navigate through a complex SAS macro or two in my day, I can assure you that it the single worst experience I have ever had in 20 years of programming.

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#46
post #32

To be honest I find the whole repeated "no, shut up" thing to be a bit crass and it makes me unsympathetic if anything. I hope this doesn't become a catchphrase in blogs.

I thought it was perfect in the original dropbox quora post, but I agree that it doesn't quite fit here, and there is certainly a danger of it becoming a meme.

The original post wasn't insightful at all. That arrogant, know-it-all attitude is not how DropBox got their interface right. They got their interface right through careful attention to their users, by being humble enough to trust the user data and throw away features they had thought would be useful.

EDIT: Downvoted, great. This must be the ultimate triumph of snark: we are now perpetuating the myth that common sense and a sassy attitude is how DropBox created a breakthrough product, instead of careful beta testing and analysis of usage data.

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#47
Theres also Tableau (http://www.tableausoftware.com/) for people interested in just pivoting data and charting. Its kind of expensive and PC only but serves that function well.

As a Statistician, I used to use SAS, Stata, R, Excel and of course SQL to extract data but for the purposes of pretty, pretty charts, Tableau is king.

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#48

Earlier quoted context omitted.

No, but I can give some suggestions. It would help to know what you want to do. First of all, you need to decide if you want a language reference, or an application guide, as R books fall into those two categories. If you have a specific type of work in mind (bio-informatics, data mining, data visualization, ...) I'd say to find a book that focuses on that topic. I haven't looked in a while, but I haven't seen a gene…

Thanks. I do biology with limited amount of data and my needs are very basic. Here is a software I wrote to do sleep analysis in Drosophila: http://www.pysolo.net So far I could satisfy most of my statistics needs with the function in numpy and scipy but occasionally I need to do something slightly more fancy and R I guess is the way to go.

Possibly. R is really great at doing "fancy" statistical analyses. It's very lousy at doing things like text manipulation. When I have a project that needs some text manipulation on the front end, I frequently use other tools (Python, vi, sed, ...) on the front end to beat text data into a nicer form for R. I couldn't say without knowing more about your project.

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#49
post #19
post #9

Earlier quoted context omitted.

Yes, I like the Python also, but you have missed the point. For MBA-types, business types, and scientists the programming concepts are too much to learn. Why should they have to learn programming when their needs are simple? It is not just "keep it simple", it is "keep it simple" for non-programmers.

I understand that point. My point was that a Python library for statistics is not the right tool for them, but that in no way makes that statistic library or Python "bad." Python is a programming language. If you think that the users you have in mind can't handle programming, then don't give them a programming language.

You are right. Python and R are not the right tool for them. They are not "dropbox" simple. Er, wait ...

Re: No, shut up. What statistical programming languages can learn from Dropbox.

#50
post #9

Earlier quoted context omitted.

Yes, I like the Python also, but you have missed the point. For MBA-types, business types, and scientists the programming concepts are too much to learn. Why should they have to learn programming when their needs are simple? It is not just "keep it simple", it is "keep it simple" for non-programmers.

Maybe I'm missing the point too, because I don't understand why he's arguing that Python and R should cater to people that don't want to use a programming language. Isn't that akin to arguing that C is too complicated because it allows you to directly access memory rather than abstracting that away? MBA- and business types have Excel. As a researcher, I flex both Python and R regulary -- but I want the full power of…

The point was if you want to help a large number of people with their statistical problems, the basics should be really simple (like dropbox).
Post reply on HN