I build a myrinet HPC cluster or two, vague memory that it was 2.5Gbit. Worked well, I particularly liked the cables (thin and optical, way nicer than IB cables). When I went to replace them the Myrinet 10G was repeatedly delayed, and seemed to be focusing on 10G ethernet, not MPI/HPC.
We ended up going with a new upstart called Pathscale, which then was bought by qlogic, and then Intel. Our benchmarks showed a clear win over Mellanox at the time. In particular on larger jobs the Mellanox drivers seemed to consume a fair bit of ram for each client you were talking to, so even 32 nodes jobs would consume a fair bit of ram. We even built a cluster out of hypertransport connected infinipath cards in a "HTX" slow instead of PCIe. Managed I believe 1.4us latency, which is still reasonably competitive today. We built SDR (first public cluster), DDR (first public cluster), QDR based clusters. But then Intel said the new Infinipath would not be supported on AMD nodes, or even previous gen Intel nodes, I said it's a non starter and switched to Mellanox. I talked to several other HPC sites that said the same thing, months later Intel killed off the product. Sad and short sited on Intel's part.