my classmate's ex-wife makes $63 an hour on the laptop. She has been without work for seven months but last month her check was $18401 just working on the laptop for a few hours. Read more here... max38.cℴm
PEP 450: Adding A Statistics Module To The Standard Library
61–70 of 85 posts
Re: PEP 450: Adding A Statistics Module To The Standard Library
#62It's not a terrible idea to support the absolute basics like mean & variance, but anything beyond that (particularly things like models or tests) is not a good idea for a standard library. Once you hit even something simple like a linear regression you have issues of how to represent missing or discrete variables, handling colinearity, or whether to do online or batch modes which can give different results. Tests in…
By that logic, you should get rid of about everything in standard libraries. A simple RNG? Anything related to floating point? Multi-threading? Character encodings? Unicode normalization? Time and date handling? All fraught with danger for those who do not know, and "basically a semester course and well out of scope for standard library docs".
Surely, it would be better to supply good implementations of algorithms rather than refrain from doing that, and letting programmers write and use bad ones instead?
IMO, the discussion should be about what c/should end up in the _standard_ library, and what is better put in a separate product/download.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#63Earlier quoted context omitted.
Some of the things I need are: - fewer dependencies for my package I've written the average() and standard_deviation() functions at least a couple of dozen times, because it doesn't make sense to require numpy in order to summarize, say, benchmark timing results. - reduced import time NumPy and SciPy were designed with math-heavy users in mind, who start Python once and either work in the REPL for hours or run non-tr…
scipy.stats distribution objects are a bit particular, that's a bit unfair to pin point them. Generally, numpy and scipy have much better docstrings than python stdlib itself.
Well, help(scipy.optimize.nonlin.Anderson) has the same problem, but you're right in that that failure mode is rare, and that numpy/scipy has good documentation. However, in the context of a stats library, I think it's okay to point out that scipy.stats has some annoying parts. ;)
In all honesty, I seldom use NumPy and rarely use SciPy, so I can't judge that deeply. I know that when I read their respective code bases I get a bit bewildered by the many "import *" and other oddities. It doesn't feel right to me. I know the reason for most of the choices - to reduce API hierarchy and simplify usability for their expected end-users - but their expectations don't match mine.
So I looked at more of the documentation. I started with scipy/integrate/quadpack.py. The docstring for quad() says, in essence, "this docstring isn't long enough, so call quad_explain() to get more documentation." I've never seen that technique used before. The Python documentation says "see this URL" for those cases.
Again, this is a difference in expectations. I argue that NumPy and Python have different end-users in mind. Which is entirely reasonable - they do! But it means that it's very difficult to simply say "add numpy to part of the standard library."
There's also a level of normalization that I would want should numpy be part of the standard library. For example, do out of range input raise ValueError or RuntimeError? scipy/ndimage/filters.py does both, and I don't understand the distinction between one or the other.
Now, in the larger sense, I know the history. RuntimeError was more common in Python, and used as a catch-all exception type. Its existence in numpy reflects its long heritage. It's hard to change that exception type because programs might depend on it.
But it means that integrating all of numpy into the standard library is not going to work: either it breaks existing numpy-based programs, or the merge inherits a large number of oddities that most Python programmers will not be comfortable with.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#64One side effect is that this would accelerate the adoption of Python 3 in the scientific community
Re: PEP 450: Adding A Statistics Module To The Standard Library
#65Earlier quoted context omitted.
Permutations and combinations already exist within the itertools module.
> Permutations and combinations already exist within the itertools module. Not exactly. Given argument lists, Itertools provides result lists (actually, iterators for that purpose) with the original elements permuted and combined, but doesn't provide numerical results for numerical arguments, as shown here: http://arachnoid.com/binomial_probability I was referring to permutation and combination mathematical functions…
Put differently, you want the functions that count the number of permutations and combinations possible, not functions that yield/generate the actual permutations and combinations.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#66Great idea, but while assembling this library, don't leave out permutations, combinations, and the binomial Probability Mass Function (PMF) and Cumulative Distribution Function (CDF). Small overhead, easy to implement, very useful. More here: http://arachnoid.com/binomial_probability
Permutations and combinations already exist within the itertools module.
There is something deep about software that thwarts our dreams of making it re-usable, and makes debates about what primitives to put into the stdlib difficult. For one thing, it seems like we should be able to collectively determine a DAG of concepts, starting from simplest primitives and progressively building up derived concepts, like the tech tree in a game like Civilization, or Principia Mathematica. But that tree is so vast, and the most-useful points so sparse along it (ex: I don't think anyone's asking for Church numerals in their stdlib) that we end up debating which are the most-useful derived concepts to surface. Moreover, there's no language or compiler that is sufficiently smart to efficiently map all those concepts to the hardware we have, we instead go with expertly-wrought implementations.
To take this example, you _could_ count enumerations and permutations by passing a range(n) list to itertools and then counting how many actual results you get back, but that's silly when you could also just use the binomial theorem to get there directly. A compiler that could generally perform such transformations would be miraculous -- well beyond the territory of automated proof assistants like mathematica or gcc -O3 that trundle along cultivated routes of expert system rules, into the realm of actually discovering deep linkages at the frontier of our knowledge.
Until then it seems like stdlibs will just fracture along lines of strain among the userbase. Presumably, most Python users don't need anything beyond what a financial calculator would provide, and anyone else should head to numpy.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#67Earlier quoted context omitted.
Technically speaking, there are no equivalent URLs in general, different strings may lead to different resources. Still, there are a number of common sense heuristics to normalize URLs, that HN applies to do de-duplication. I was wondering what is the rationale for not having trailing slash removal among them. I mean, is there any legitimate website that serves a different resource if you remove the trailing slash?
Per RFC3986/7, http://example.com/%60 and http://example.com/a are equivilant. (Indeed, all major browsers will request the latter regardless of what is input.) Equally, punycode encoded IRIs and the original IRI are equivilance. There is a whole section on equivilance in both of the RFCs (3967 includes 3986 by reference, so is a superset).
Browser equivalence is another thing entirely. Most browsers will accept http://www。google。com (because in Japanese '。' is '.'). But if you tried to request that actual resource it doesn't lead anywhere.
But yeah HN should just use browser equivalence.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#68Earlier quoted context omitted.
They probably left out pandas because it depends on numpy and also this point: > For many people, installing numpy may be difficult or impossible. that's as true, and arguably more, for pandas.
Although the solution, imho, is to make numpy easier to install across many platforms (ideally `pip install numpy pandas` should just work).
There are two types of potential limitations to installing software, technical and policy. Just solving the technical won't necessarily make it easier to install numpy/pandas on a users computer from a policy perspective. Being able to rely on basic statistics functions being available in a default python install would be nice.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#69> For many people, installing numpy may be difficult or impossible. For example, people in corporate environments may have to go through a difficult, time-consuming process before being permitted to install third-party software. I do not regard this as a good justification for putting something in the standard library! If you don't have root access, use vitualenv (which you might want to do anyway) and install the pa…
Not everybody using python is writing freestanding software for machines they control. Lots of software for example ship with a python install for scripting, automation or writing plugins for example.
Re: PEP 450: Adding A Statistics Module To The Standard Library
#70Just out of curiosity, I submitted this yesterday: https://news.ycombinator.com/item?id=6190603 The URL was http://www.python.org/dev/peps/pep-0450/ While this is http://www.python.org/dev/peps/pep-0450 That is, exactly the same except for a trailing slash. Doesn't the deduplication algorithm handle this case?
Why should it?
For one, it provides the welcome ability to bring topics up in Hacker News again, where they might get accepted better the second or third time (e.g because more people are online at the time of the second submission).
If the "deduplication algorithm" had "handled this case", then we would only be left with the first submission (a dead discussion), whereas as it is, HN users have now caught on to this PEP news and we have a discussion going on.