Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Help us collect modern Python NumPy code solutions

github.com

Post reply on HN
Print view View original

21–30 of 41 posts

Re: Help us collect modern Python NumPy code solutions

#21

Post by m00dy »

PartiallyTyped wrote: post #9
m00dy wrote: post #4

#write a function that zips numpy arrays import numpy as np def zip_arrays(arrays): return np.array([np.concatenate(arr) for arr in zip(*arrays)])

>>> arrays = [np.arange(1000)]*10 >>> zip_arrays(arrays) Traceback (most recent call last): File " ", line 1, in File " ", line 2, in zip_arrays2 File " ", line 2, in File " ", line 180, in concatenate ValueError: zero-dimensional arrays cannot be concatenated Try this: def zip_arrays(arrays): return np.concatenate([array[:l,...,None] for l in [min(array.shape[0] for array in arrays)] for array in arrays], axis=-1) O…

well, I used the code from Copilot :)

Re: Help us collect modern Python NumPy code solutions

#22

Post by danuker »

johndough wrote: post #2

For almost all issues, you can simply search for site:stackoverflow.com + the title of the issue and get a larger variety of solutions ranked by upvotes, so this repository seems strictly worse. Is there any advantage that this repository brings over just searching for the question on StackOverflow?

Even better, search the stackoverflow.com site directly. It seems Google no longer indexes everything (if it ever did).

Re: Help us collect modern Python NumPy code solutions

#23

Post by danbruc »

kingaillas wrote: post #20
xnorswap wrote: post #15

Earlier quoted context omitted.

Indeed, their example is terrible, because it's also the kind of 'toy' problem that just doesn't have good real world uses too. How often do you want to reverse a string. I think your solution is fine and isn't "bad code" at all. Certainly a real explanation warrants talking about the IEnumerable interface. I'd also consider the simpler (and ever so slightly better performing): var buffer = "1234simple".ToArray(); Ar…

> I can't understand the use-case of needing to do this outside of toy problems. Not sure, not my domain, but perhaps bioinformatics has the equivalent of the convolution operation on sequences? EDIT: there is the "reversal" operation. Peeking ahead from the Rosalind problems, I'm not quite up to Reversal Distance ( https://rosalind.info/problems/rear/ ).

But would they represent their data as character strings or would they pack four nucleotides with two bits each into one byte? Or would they be working on the amino acid level or something else all together? And would they really physically reverse potentially very long strings or just use a decreasing index into the original string? I am sure reversing strings has some use cases, maybe even in bioinformatics, but I would agree with the other comment, it seems a pretty niche operation, otherwise there would probably be a String.Reverse() method.

Re: Help us collect modern Python NumPy code solutions

#24

Post by johndough »

Beltiras wrote: post #14
johndough wrote: post #12

Earlier quoted context omitted.

The code quality seems questionable. A few examples: Repository claims modern code, but uses Python 2 (end of life January 1, 2020) https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Copied from https://wiki.python.org/moin/Powerful%20Python%20One-Liners Incorrect indentation causing unnecessarily large memory usage https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Undefined behavior for i…

Fixed some of those. Simple case of just pushing the edit button and sending a PR.

The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files.

I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array.

But there are still many other issues like using default platform string encoding instead of detecting it properly or at least using utf-8, checking for ".txt" anywhere in the path instead of at the end, and not closing the input files with a context manager like the output file, which suggests that this code is just pierced together from various sources.

A robust solution would require many more lines.

Re: Help us collect modern Python NumPy code solutions

#25

Post by PartiallyTyped »

johndough wrote: post #18
PartiallyTyped wrote: post #9

Earlier quoted context omitted.

>>> arrays = [np.arange(1000)]*10 >>> zip_arrays(arrays) Traceback (most recent call last): File " ", line 1, in File " ", line 2, in zip_arrays2 File " ", line 2, in File " ", line 180, in concatenate ValueError: zero-dimensional arrays cannot be concatenated Try this: def zip_arrays(arrays): return np.concatenate([array[:l,...,None] for l in [min(array.shape[0] for array in arrays)] for array in arrays], axis=-1) O…

I think a more Pythonic solution would be np.array(arrays).T or maybe np.copy(arrays).T for one less character. The shortest solution is probably np.c_[arrays].T but likely causes a Google search on first read.

That does not zip though. In fact, it creates a 1-dimensional array of objects because the arrays do not have the same number of items.

     >>> a = [np.arange(i) for i in range(1, 11)]

     >>> np.array(a).T

        array([array([0]), array([0, 1]), array([0, 1, 2]), array([0, 1, 2, 3]),

        array([0, 1, 2, 3, 4]), array([0, 1, 2, 3, 4, 5]),

        array([0, 1, 2, 3, 4, 5, 6]), array([0, 1, 2, 3, 4, 5, 6, 7]),

        array([0, 1, 2, 3, 4, 5, 6, 7, 8]),

        array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])], dtype=object)
In fact, if we are to zip them, it should yield:

>>> list(zip(*a))

[(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)]

zip_arrays otoh does give the same result.

>>> zip_arrays(a)

array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])

Re: Help us collect modern Python NumPy code solutions

#26

Post by RodgerTheGreat »

johndough wrote: post #17
hoosieree wrote: post #10

I learned J before NumPy and I'm glad I did, because it's much easier to see and learn the algorithms when they're short symbols. Compare np.sqrt(sum(np.square(x))) versus %: +/ *: x So if you can translate one of the array languages to NumPy, you can tap into the wealth of "idioms" collected over the years. For example: J Phrases[0] (organized by category), BQNcrate[1] and APLcart[2] (searchable). [0]: https://www.j…

I think a fairer comparison would be np.linalg.norm(x) or even norm(x) if you import numpy.linalg.norm as norm beforehand. This comes with the advantage that you can look up the function by name to know what it does instead of having to look up chains of symbols on several websites.

In practice, if you're reading a J expression which contains unfamiliar primitives, there's only one place you need to look: https://code.jsoftware.com/wiki/NuVoc

Idioms are composed from primitives, but many of them are shorter than a descriptive name would be, and since the set of primitives is small (compared to the standard library in most languages), it doesn't take that long to memorize most or all of the symbols.

Re: Help us collect modern Python NumPy code solutions

#27

Post by mynameismon »

johndough wrote: post #24
Beltiras wrote: post #14

Earlier quoted context omitted.

Fixed some of those. Simple case of just pushing the edit button and sending a PR.

The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files. I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array. But there are still many other issues like using default platform string encoding instead of d…

Another memory issue (although unlikely) could also be the file itself being larger than the RAM size, which can be avoided by chunking the textfiles.

Re: Help us collect modern Python NumPy code solutions

#28

Post by omarhaneef »

Just had a quick look at some of the responders who collectively say:

1 What is the use of this curated list over just accepting the popularly voted answer?

2 I found some errors in the default answer and here is a detailed explanation of what was wrong with the answer.

And I hope they can see that #2 shows the issues with #1.

To put it another way if I were looking for the best solution (which I often am), and I had a choice between a one line stack overflow answer and one of these detailed solutions, I would pick the detailed, curated solution every time.

edit: And the best stack overflow answers are often detailed curated solutions, so I am not saying they don't wind their way up there.

Re: Help us collect modern Python NumPy code solutions

#29

Post by johndough »

PartiallyTyped wrote: post #25
johndough wrote: post #18

Earlier quoted context omitted.

I think a more Pythonic solution would be np.array(arrays).T or maybe np.copy(arrays).T for one less character. The shortest solution is probably np.c_[arrays].T but likely causes a Google search on first read.

That does not zip though. In fact, it creates a 1-dimensional array of objects because the arrays do not have the same number of items. >>> a = [np.arange(i) for i in range(1, 11)] >>> np.array(a).T array([array([0]), array([0, 1]), array([0, 1, 2]), array([0, 1, 2, 3]), array([0, 1, 2, 3, 4]), array([0, 1, 2, 3, 4, 5]), array([0, 1, 2, 3, 4, 5, 6]), array([0, 1, 2, 3, 4, 5, 6, 7]), array([0, 1, 2, 3, 4, 5, 6, 7, 8])…

You are completely right. Nevertheless, when I zip over arrays of different length, it is usually a bug instead of being intentional.

A code-golfed version for different array lengths:

    np.c_[[a[:min(map(len,arrays))].T for a in arrays]].T
The trick to get a local variable within a list comprehension statement with

    for l in [foo()]
from your zip_array function is very cool by the way, but I sacrificed it in favor of shorter code.

Re: Help us collect modern Python NumPy code solutions

#30

Post by kortex »

infinityio wrote: post #7

Just had a quick look at a few of the python examples: - it might be neat, but advocating for `eval(input())` [0] might not be the safest solution for this problem, especially without explaining the dangers of `eval` (assuming this site is partially aimed at beginners?) - for an article titled 'how to terminate a script', the suggested method (`quit()`) [1] is specifically described in the official python docs [2] as…

I agree eval is pretty gross and folks should not be encouraging its use since it can lead to bad habits.

Quit/exit are pretty innocuous though. I actually just thought exit was an alias for sys.exit. I've never had a problem using plain exit() in programs.

Post reply on HN