Viewing profile — antinucleon

Yes

Mojo is trying to create a new language to solve the problem, and specialized for CPU. We are using a more pragmatic way to solve GPU AI computation problem.

We haven't compared yet.

We developed AITemplate majorly for Meta's focus at that time, eg Ads/Ranking need. For HippoML is startup we are building for Generative AI. HippoML is not using AITemplate.

It is actually non-trivial to get GPU run fast, especially on SoC with strong CPU like M2.

Hippo is faster than AITemplate, and supports more generative models. We haven't compared vs TVM, but for absolute token/s on M2 Max, Hippo is able to run decoding on LLAMA with da…

We will disclose more details very soon.

Yes. We support >= 1bit <= 16bit models out of box for various of models.

AITemplate's original designer is here. We quit Meta in January and start HippoML ( https://hippoml.com/ ). We just disclosed our new engine's performance on LLM: https://blog.hipp…

CuDNN v7 was used in the experiments, in the experiments parts each comparison was listed with version or commit number.

Summary: Tensor program is able to be optimized by using machine learning and transfer learning. The numerical program optimization model is trained on feature from low-level AST o…

The version tested in paper should not have P2P,so it was much slower than current version.

There is a huge distributed performance advantages vs TensorFlow. You can get a hint from Prof. Carlos Guestrin's keynote talk at Data Science Summit 2016. Also, CMU CS Dean Andrew…