Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Visual Intro to NumPy and Data Representation

jalammar.github.io

Post reply on HN
Print view View original

11–20 of 23 posts

Re: A Visual Intro to NumPy and Data Representation

#12

Post by ovi256 »

quietbritishjim wrote: post #10
lejar wrote: post #7

Nice overview! One thing I think you should add, which I find immensely useful is the reordering of arrays using indexing. Take for example: In [2]: numpy.array([1, 2, 3])[[0, 2, 1]] Out[2]: array([1, 3, 2]) You index using a list and it gives you a view of the array with the new order (the underlying array is not changed and there is no copy being done).

Using "fancy" indices like this does result in a copy because it can't be represented as a simple slice of the original matrix. A good explaination is here (it's from 2008 but still true): https://scipy-cookbook.readthedocs.io/items/ViewsVsCopies.ht... You can verify there's a copy by changing the new array after putting the result in a new variable (see above link for why this makes a difference) and verifying the o…

A copy-on-write mechanism triggered by `y[0] = 3` would look the same and pass the test you devised, so you can't eliminate the possibility that it exists.

A better way would be to track memory use. A copy being created by either `y = x[[0, 2, 1]]` or `y[0] = 3` would show as a memory increase.

Re: A Visual Intro to NumPy and Data Representation

#15

Post by jcims »

lejar wrote: post #7

Nice overview! One thing I think you should add, which I find immensely useful is the reordering of arrays using indexing. Take for example: In [2]: numpy.array([1, 2, 3])[[0, 2, 1]] Out[2]: array([1, 3, 2]) You index using a list and it gives you a view of the array with the new order (the underlying array is not changed and there is no copy being done).

As an aside, one of my major challenges grokking numpy and pandas is the semantically dense syntax like the above. I know that the layers of bracing have an impact but it's difficult for me to tell where it is applied and/or described.

Re: A Visual Intro to NumPy and Data Representation

#17

Post by improbable22 »

iandanforth wrote: post #16

It would be good to mention the @ operator in the matrix multiplication section. https://alysivji.github.io/python-matrix-multiplication-oper...

A warning sign that your faith in 0-based indexing may be faltering -- catching yourself writing comments like this :)

    # element at the top right. i.e. (1, 2) aka (0, 1) in python
    A[0, 0] * B[0, 1] + A[0, 1] * B[1, 1]

Re: A Visual Intro to NumPy and Data Representation

#18

Post by improbable22 »

quietbritishjim wrote: post #10
lejar wrote: post #7

Nice overview! One thing I think you should add, which I find immensely useful is the reordering of arrays using indexing. Take for example: In [2]: numpy.array([1, 2, 3])[[0, 2, 1]] Out[2]: array([1, 3, 2]) You index using a list and it gives you a view of the array with the new order (the underlying array is not changed and there is no copy being done).

Using "fancy" indices like this does result in a copy because it can't be represented as a simple slice of the original matrix. A good explaination is here (it's from 2008 but still true): https://scipy-cookbook.readthedocs.io/items/ViewsVsCopies.ht... You can verify there's a copy by changing the new array after putting the result in a new variable (see above link for why this makes a difference) and verifying the o…

A related fun fact, when slicing several dimensions:

    >>> a = np.arange(9).reshape(3,3) # a matrix
    >>> a[0:3,0:3]          # ranges are treated independently
    array([[0, 1, 2],
           [3, 4, 5],
           [6, 7, 8]])
    >>> a[[0,1,2],[0,1,2]]  # but arrays are treated at once
    array([0, 4, 8])

Re: A Visual Intro to NumPy and Data Representation

#19

Post by iandanforth »

improbable22 wrote: post #17
iandanforth wrote: post #16

It would be good to mention the @ operator in the matrix multiplication section. https://alysivji.github.io/python-matrix-multiplication-oper...

A warning sign that your faith in 0-based indexing may be faltering -- catching yourself writing comments like this :) # element at the top right. i.e. (1, 2) aka (0, 1) in python A[0, 0] * B[0, 1] + A[0, 1] * B[1, 1]

That's called the "Matlab Hangover"
Post reply on HN