Earlier quoted context omitted.
Also, custom ASICs are the current state of the art for NN. edit: missing word
Which custom ASICs are you talking about?
I'm referring to Google's TPU.
A Tiny Chip That Could Disrupt Exascale Computing (2015)
51–57 of 57 posts
Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#52Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#53Earlier quoted context omitted.
Just to clarify: are you trying to compete with Nvidia, or with Intel? If you're going against GPUs, is your chip something that can run neural networks (better than Nvidia)?
Short answer: If we were to implement SIMD FP16 support similarly to how we have a planned dual FP32 in our FP64 FPU, we would be able to easily match GPU performance by throwing more cores at the problem, while still being more efficient. While neural nets/machine learning is interesting, and we could potentially enable it in new forms as we can provide a desktop GPU's capability in a much smaller/lower power form f…
Interesting. Which applications require vector-vector or matrix-vector operations as opposed to matrix-matrix?
Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#54Earlier quoted context omitted.
I'm referring to Google's TPU.
I couldn't find any details about that chip. How do you know it's state of the art?
I only know what is publicly know. It was discussed a while ago on HN. I think that google claims the best performance per Watt and manages to do that with specialized 8 bit floating point ALUs. I don't think it has been publicly available yet so the claims lack third party verification.
Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#55Earlier quoted context omitted.
Did you stick with the parallel, SERDES-less interfaces for your interchip I/O? 48 GB/s implies a pretty high signalling rate to not have a CTLE, DFE, etc. Why 3 interchip links? What network topology are you planning to use to scale to large numbers of chips? If you're still using parallel I/O, how are you planning to communicate beyond a single PCB? What memory interface are you using? The article seems to confuse…
We have partnered with a startup (we'll announce who soon enough) who shared a lot of ideas about chip to chip I/O with myself. While they call it a SerDes, it is infact a source synchronous (clock forwarded) link that is 5 bits over 6 wires. It is silicon proven, and is capable of up to 125Gb/s over 12mm while being a little over 10x more energy efficient (in terms of pJ/bit) than other available VSR SerDes. Obvious…
25 gbps for a very short reach interconnect sounds possible, although having to go through an adapter chip is going to kill your latency from a system perspective. If you haven't already, you should check out the DE Shaw Research Anton 2 chip. It is an older process, but it has 66 4-way processor cores running at 1.65 Ghz and a roughly comparable network (although 6-way rather than 4-way), in addition to all of the md-specific hardware. It uses a similar memory hierarchy (although it does use non-coherent caches). Getting good performance out of software managed caches is very difficult in practice, even if you know your problem extremely well. With very carefully written software (and a sufficiently friendly problem) good performance is possible, but it definitely isn't easy.
Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#56Earlier quoted context omitted.
From what I understood, a lot of the software stack would require rewriting. As it is, it doesn't look like it would be friendly to a Linux environment running natively on it, but could be more amenable to a coprocessor-like environment where the host would load programs and the Neo would run them. Did I get it right?
In the near term, yes, though that is primarily a business reason for us. Supporting Linux is technically possible (old projects such as uCLinux were built around running on MMU-less systems like ours; Mainline Linux 4.2 started to have limited support for a couple of MMU-less systems), though our target areas (HPC and DSP-like tasks) don't necessarily need anything more than a microkernel/RTOS. A full OS like Linux…
I am perfectly fine with the idea to have a supercomputer running a specialized OS and a front-end machine running the sysadmin-friendly OS. It feels like a Connection Machine with fewer blinking lights.
Re: A Tiny Chip That Could Disrupt Exascale Computing (2015)
#57Founder of REX here, and surprised to see this posted here. Happy to answer any questions, and you can check my comment history for some of my prior posts on REX. We've had some really great progress that we hope to share in the near future, so stay tuned. EDIT: Since this article is over a year old, we have made a lot of progress, and have recently taped out our first chip. We haven't officially posted a job opening…
Hey Thomas, I actually applied for a hardware position a couple of days ago, but I'm interested in software as well (and a mix of the two)!
Think you're doing some very exciting work with REX and would love to be a part of the team :).