21–30 of 34 posts
One thing that is holding me back in numpy is not knowing the runtime complexity of operations—of course I can profile code, but I should have better awareness when writing code in the first place. Without an algorithms background, I don't have strong intuitions on the runtime complexity of the primitives (np.unique). Any suggestions?
the suggestion that jumps out is to just learn about algorithms.
Does anyone know of any similar resources for Pandas? I've found the following to be quite helpful but would love to know if anyone knows of other resources in a similar vein: https://pandas.pydata.org/pandas-docs/stable/cookbook.html
One thing that is holding me back in numpy is not knowing the runtime complexity of operations—of course I can profile code, but I should have better awareness when writing code in the first place. Without an algorithms background, I don't have strong intuitions on the runtime complexity of the primitives (np.unique). Any suggestions?
what an odd hangup. how does this have anything to do with numpy specifically? the suggestion that jumps out is to just learn about algorithms.
One thing that is holding me back in numpy is not knowing the runtime complexity of operations—of course I can profile code, but I should have better awareness when writing code in the first place. Without an algorithms background, I don't have strong intuitions on the runtime complexity of the primitives (np.unique). Any suggestions?
What other library tells you about complexity? And as you tell, if you don't know algorithns well, I'm pretty sure your implementations won't have better complexity.
For example from std::vector::insert [1]:
Complexity
1-2) Constant plus linear in the distance between pos and end of the container.
3) Linear in count plus linear in the distance between pos and end of the container.
4) Linear in std::distance(first, last) plus linear in the distance between pos and end of the container.
5) Linear in ilist.size() plus linear in the distance between pos and end of the container.
edit: formatting
Working through them and noticed a few small things. For #3, you can make a boolean array with np.ones/np.zeros with the same dtype arg, saves a little bit of space. ie np.ones((3,3), dtype=bool) For #14, you can make use of the same compound boolean statements as you can in pandas to make it a bit simpler. ie a[(a > 5) & (a For #15, this is a built in numpy function. np.maximum(a,b). That's as far as I've made it, b…
However, for No. 15, that is not the point of the exercise.
Earlier quoted context omitted.
Switch to Julia! Hit @edit unique([1,2,3,2]) in the REPL and you see the implementation.
Nice! Ive been meaning to try out Julia for a while now. Is the numpy equivalent in Julia largely written in Julia itself? (as opposed to C/Fortran)
One thing that is holding me back in numpy is not knowing the runtime complexity of operations—of course I can profile code, but I should have better awareness when writing code in the first place. Without an algorithms background, I don't have strong intuitions on the runtime complexity of the primitives (np.unique). Any suggestions?
What other library tells you about complexity? And as you tell, if you don't know algorithns well, I'm pretty sure your implementations won't have better complexity.
For example, np.einsum for all its greatness in the past wasn't faster than np.tensordot, but it was more flexible. One can tell einsum to try and use the same underlying BLAS functions that tensordot uses (which can parallelise the computation) if applicable, and it will likely be default for einsum to perform this optimisation automatically once the devs iron out some bugs. But for now, it pays to know how the two methods are different.
One thing that is holding me back in numpy is not knowing the runtime complexity of operations—of course I can profile code, but I should have better awareness when writing code in the first place. Without an algorithms background, I don't have strong intuitions on the runtime complexity of the primitives (np.unique). Any suggestions?
Switch to Julia! Hit @edit unique([1,2,3,2]) in the REPL and you see the implementation.
np.unique??
(Also numpy has some really nice features over Matlab, like [None,:] broadcasting and being able to index a parenthesized expression or function output without naming it. Ok, the latter is not really a feature, more of an example of how Matlab is broken as a language)