Viewing profile — Marat_Dukhan

Linux-capable RISC-V cores often have 64-bit architecture and no SIMD/vector processing capabilities.

IMO the author alludes to some enterprise software running on Wintel that has per-core licensing costs.

He's a Software Engineer on the TPU team. Are you confusing him for Thomas Kurian, GCloud SVP? Note: I work for Google, but speak for myself.

If by acceleration you mean offloading inference to a different IP block (GPU/DSP/NPU), then yes. XNNPACK is the inference engine for CPU. CPU is the default backend in TensorFlow …

In order to benefit from optimizations in *this blog post* the model needs to be quantized to 8-bit integers. However, XNNPACK supports floating-point inference as well (including …

It performs fixed-point arithmetic on 8-bit integers. You can mimick lower than 8-bit precision by using output_min/output_max parameters in XNNPACK operators, but keep in mind tha…

Yes, these optimizations work with existing tflite models, so long as the quantized operators they use are supported in XNNPACK.

TensorFlow doesn't support quantized inference (it supports only mimicking quantization in floating-point for quantization-aware training), so it can't immediately benefit from the…

Author here, happy to take your questions.

Good. I was surprised that Apple Silicon Macs don't have a built-in cellular modem. This reminds me how iPhone launched without a 3G modem, and I hope Apple will similarly fix the …

Even WebGL2 doesn't expose compute shaders, so any NN computations work by abusing the graphics pipeline, with many inefficiencies involved. Shader dispatch is more expensive, no a…

What a time to be alive!

Performance on the plot is higher than FP32 peak, but there's no error - because FBGEMM does not compute in FP32, it computes in 8-bit fixed point. On a Broadwell CPU, you can do 1…

FBGEMM is faster than theoretical peak FP32 (single-precision floating-point) performance, therefore its faster than SGEMM/DGEMM in any BLAS library

QNNPACK directly competes with the CPU backend of TensorFlow Lite and the gemmlowp library. The Caffe2 backend of PyTorch 1.0 integrates QNNPACK, and directly competes with TensorF…

You can use the same toolchain to convert PyTorch model to Caffe2 through ONNX. Caffe2 supports both Android and iOS. There is even a tutorial: http://pytorch.org/tutorials/advance…

It is possible to perform some computations using OpenGL ES 3.0 / WebGL 2.0, but many types of operations (e.g. anything that involves random-access writes) are impossible, and man…

WebGL 2 is based on OpenGL ES 3.0, it doesn't give you compute. Compute shaders were added in OpenGL ES 3.1