To be honest I find the whole repeated "no, shut up" thing to be a bit crass and it makes me unsympathetic if anything. I hope this doesn't become a catchphrase in blogs.
No, shut up. What statistical programming languages can learn from Dropbox.
21–30 of 58 posts
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#22The code that makes him say "what a mess," I think is beautiful: def summary(data, key=itemgetter(0), value=itemgetter(1)): for k, group in groupby(data, key): yield (k, sum(value(row) for row in group)) Perhaps that's because I'm a programmer, and Python is a general purpose programming language. But I think that's what his complaint boils down to: the Python statistical code looks too much like Python . Which, yeah…
I think his point here is that the most visible aspects of the code are the structures built up to do the computation, rather than the computation itself. As a description of a generator loop, it reads quite nicely. But the language does not give much ground to the topic it's describing, in the way (to use the obvious example) Lisp would. I think that's what he is getting at.
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#23The code that makes him say "what a mess," I think is beautiful: def summary(data, key=itemgetter(0), value=itemgetter(1)): for k, group in groupby(data, key): yield (k, sum(value(row) for row in group)) Perhaps that's because I'm a programmer, and Python is a general purpose programming language. But I think that's what his complaint boils down to: the Python statistical code looks too much like Python . Which, yeah…
[deleted]
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#24Earlier quoted context omitted.
There are GUI statistics apps for people who just want the common case, Dropbox-style: packages like Weka for data mining / predictive statistics, SPSS for descriptive statistics, and a dozen other such things. The statisticians who choose to use a programming language like R or Python typically do it because they actually do want a programming language. I mean, that's why Bell Labs statisticians invented S (R's pred…
I am a statistician that does both research and applied work. I use R for three reasons: (1) It's Free Software; (2) It's a programming language; (3) Other statisticians use it so it's easier for me to collaborate. There are the usual supporting arguments for (1). (2), I've only used SAS a little bit, and it was extremely unpleasant to use it for non-built-in stuff, which makes research harder for no good reason. For…
can you reccomend a book to get started with r?
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#25Earlier quoted context omitted.
Yes, I like the Python also, but you have missed the point. For MBA-types, business types, and scientists the programming concepts are too much to learn. Why should they have to learn programming when their needs are simple? It is not just "keep it simple", it is "keep it simple" for non-programmers.
Maybe I'm missing the point too, because I don't understand why he's arguing that Python and R should cater to people that don't want to use a programming language. Isn't that akin to arguing that C is too complicated because it allows you to directly access memory rather than abstracting that away? MBA- and business types have Excel. As a researcher, I flex both Python and R regulary -- but I want the full power of…
Agree with this point. I'm both MBA/bizdev and software engineer. When putting on my MBA hat and working on sales forecast, decision making models, spread sheet is all I use. It is quick, tweakable, super easy to share. Whereas building my site which focuses on market research services, I resorted to C and existing stats packages cause they are powerful, more flexible, and basically programmable. To me what MBA/bizdev people need is significantly different from what a software developer writing stats-related code need. It is a very different scenario from the dropbox story...
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#26There are quite a few really nice tools out there for this kind of data analysis (generally called Business Intelligence, or BI for short). None of them that I have found are procedural, they are all based on interactive dashboards. I'm sure that there is some scripting or XML formatting required behind the scenes to get the system set up to accept data, but after that it's all point-and-click.
The systems that I've seen/evaluated are Needlebase, Birst and Spotfire. None of them are particularly cheap, but if you're in a business where real-time access to data would help your team make better decisions, they could be very valuable.
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#27I've been using http://tablib.org/ for awhile now to read in tabular data. With this summary fn and a few other functions to simplify the process of aggregating data into useful views I think you've got a winning solution to the author's complaint.
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#28Earlier quoted context omitted.
I am a statistician that does both research and applied work. I use R for three reasons: (1) It's Free Software; (2) It's a programming language; (3) Other statisticians use it so it's easier for me to collaborate. There are the usual supporting arguments for (1). (2), I've only used SAS a little bit, and it was extremely unpleasant to use it for non-built-in stuff, which makes research harder for no good reason. For…
can you reccomend a book to get started with r?
No, but I can give some suggestions. It would help to know what you want to do.
First of all, you need to decide if you want a language reference, or an application guide, as R books fall into those two categories.
If you have a specific type of work in mind (bio-informatics, data mining, data visualization, ...) I'd say to find a book that focuses on that topic. I haven't looked in a while, but I haven't seen a general R book that I like, anything I suggest there would be guessing on my part.
There are plenty of good references on the web. I'd start by looking at the material available from the R web site:
R's core manuals [1] are typically correct and reasonable to use. The "Introduction to R" guide will get you up to speed fairly well if you already know another programming language. There is also the contributed documentation [2]. I haven't gone through these, so I can't say much about them, or promise that they are up-to-date. I suspect not, as R develops rapidly. The one reference I can recommend highly is "The R Inferno" by Patrick Burns [3]. This is not a starter guide, but something you read after one. It gives excellent advice on avoiding common pitfalls in R.
[1] http://cran.r-project.org/manuals.html
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#29The code that makes him say "what a mess," I think is beautiful: def summary(data, key=itemgetter(0), value=itemgetter(1)): for k, group in groupby(data, key): yield (k, sum(value(row) for row in group)) Perhaps that's because I'm a programmer, and Python is a general purpose programming language. But I think that's what his complaint boils down to: the Python statistical code looks too much like Python . Which, yeah…
Picking on Python for not having simpler built-in ways to do domain-specific statistical operations seems rather silly to me.
I've been involved the last few years with creating better data structures and tools for doing statistics in Python-- with excellent results (http://pandas.sourceforge.net and http://statsmodels.sourceforge.net). So I think the author should take a closer look at some of the libraries and tools out there.
Re: No, shut up. What statistical programming languages can learn from Dropbox.
#30There's a nice NumPy-based Python package called Tabular (http://www.parsemydata.com/tabular/index.html) that makes this super easy:
import numpy as np
import tabular
# CSV with Region, City and Sales columns
data = tabular.tabarray(SVfile = 'data.csv')
# Calculate the total sales within each region
summary = data.aggregate(On = ['Region'], AggFuncDict = {'Sales':np.sum}, AggFunc = len)
summary.saveSV('summary.csv')