Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
1–10 of 50 posts
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#2The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It would be more ergonomic if each tensor had labeled dimensions. That would prevent the kind of silly mistake where you mix up the ordering of dimensions and only notice it when you later change the shape of the tensors so the different dimensions no longer match up.
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#3https://www.theatlantic.com/magazine/archive/2017/04/why-is-...
https://www.newsweek.com/2016/06/17/silicon-valley-takeover-...
https://medium.com/@preethikasireddy/why-im-leaving-silicon-...
https://getlighthouse.com/blog/silicon-valley-bad-managers/
https://www.philanthropy.com/article/A-Star-Performer-Create...
https://www.theglobeandmail.com/report-on-business/internati...
https://www.geekwire.com/2018/emily-chang-brotopia-silicon-v...
https://www.entrepreneur.com/article/297481
https://thenextweb.com/insider/2017/03/11/bro-culture-poison...
https://www.quora.com/Is-the-culture-in-the-Silicon-Valley-b...
https://www.vanityfair.com/news/2016/06/ellen-pao-memoir-sil...
https://www.recode.net/2018/2/5/16972096/emily-chang-brotopi...
https://www.glassdoor.com/Reviews/Silicon-Valley-Community-F...
https://www.wired.com/story/everyone-hates-silicon-valley-ex...
https://www.intelligencesquaredus.org/debates/silicon-valley...
https://yourstory.com/2017/07/silicon-valley-bro-culture/
https://www.wsj.com/articles/the-quiet-efforts-to-battle-sil...
https://thebolditalic.com/a-source-of-sorrow-in-silicon-vall...
https://www.bizjournals.com/sanfrancisco/news/2019/01/25/is-...
https://www.rawstory.com/2018/07/silicon-valley-become-next-...
https://builttoadapt.io/so-you-want-to-build-a-silicon-valle...
https://www.youtube.com/watch?v=h3-bMDaMhLg
https://www.theguardian.com/world/2017/mar/01/silicon-valley...
https://www.indiewire.com/2018/07/silicon-valley-actress-ali...!
https://www.theneweconomy.com/technology/silicons-sexist-bro...
https://news.ycombinator.com/item?id=17051800
https://www.losaltosonline.com/news/sections/schools/210-sch...
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#4The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It would be more ergonomic if each tensor had labeled dimensions. That would prevent the kind of silly mistake where you mix up the ordering of dimensions and only notice it when you later change the shape of the tensors so the different dimensions no longer mat…
I think, this is something that the community is now slowly realizing. I've must have read about that multiple times recently.
Some packages that seem to provide tensors with labeled dimensions.(I haven't personally worked with them):
xarray (Python): https://github.com/pydata/xarray NamedArrays.jl (Julia): https://github.com/davidavdav/NamedArrays.jl
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#5The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It would be more ergonomic if each tensor had labeled dimensions. That would prevent the kind of silly mistake where you mix up the ordering of dimensions and only notice it when you later change the shape of the tensors so the different dimensions no longer mat…
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#6I remember when I was learning matrix calculus and realized at some point that it was much simpler to convert everything to index notation, perform all operations, then convert everything back to standard notation at the end. It became almost comically simple, because you're "just" working with labeled scalars at that point. To be fair, it's convenient to memorize some of the more commonly used expressions (like ∂tr(AB)/∂B) rather than rederive them from scratch.
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#7Looks like this is a re-post of the same link from this somewhat recent post: https://news.ycombinator.com/item?id=16986759
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#8I've never really understood the point of Einstein notation, as a piece of mathematical notation. Is writing something like A[i, j] * B[j, k] really that much faster than writing something like Sum[j](A[i, j] * B[j, k])? Especially when you have to check the left hand side of the equality sign just to know which indices to sum over, it seems like making things less clear for a minuscule saving on ink.
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#9The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It would be more ergonomic if each tensor had labeled dimensions. That would prevent the kind of silly mistake where you mix up the ordering of dimensions and only notice it when you later change the shape of the tensors so the different dimensions no longer mat…
[deleted]
Re: Einsum Is All You Need – Einstein Summation in Deep Learning (2018)
#10I've never really understood the point of Einstein notation, as a piece of mathematical notation. Is writing something like A[i, j] * B[j, k] really that much faster than writing something like Sum[j](A[i, j] * B[j, k])? Especially when you have to check the left hand side of the equality sign just to know which indices to sum over, it seems like making things less clear for a minuscule saving on ink.
Yes, it's incredibly faster. You do a lot of tensor algebra in a relativity class.