Viewing profile — darkolorin

Hi guys. We made own Mac app for M series hardware. Based on own engine called uzu (available with MIT on GitHub). Completely written from scratch and inference too. Our goal is to…

Basically “faster” means better performance e.g. tokens/s without loosing quality (benchmarks scores for models). So when we say faster we provide more tokens per second than llama…

We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support …

I made it! 90 t/s on my iPhone with llama1b fp16 We completely rewrite the inference engine and did some tricks. This is a summarization with llama 3.2 1b float16. So most of the t…

It's my first experience on Medium. I hope community can help me to improve my skills.

really great idea, but why morning? %)