71–72 of 72 posts
Earlier quoted context omitted.
Infiniband used to be great but in an era of 40 or 100 gig ethernet it's not particularly special anymore.
Right. As https://en.wikipedia.org/wiki/InfiniBand#cite_note-20 says "2009: of the top 500 supercomputers in the world, Gigabit Ethernet is the internal interconnect technology in 259 installations, compared with 181 using InfiniBand.[20]"
This might be a good time to ask: my main reservation about TensorFlow is that it's a subset of general purpose computing, so will always be limited to niches like AI or physics simulations or protein folding. If we look at something like MATLAB (or GNU Octave) as general-purpose vector computing, then we need some kind of bridge between the two worlds. I couldn't find much other than this: https://www.quora.com/How-…
I don't know about tensor flow in particular but are little-known methods of running "general purpose" parallel programs on GPUs. Specifically, H. Dietz' MOG, "Mimd on GPU". It's a shame the project hasn't gotten more attention imo. http://aggregate.org/MOG/ See: https://en.wikipedia.org/wiki/Flynn%27s_taxonomy for explanations of terms.