Viewing profile — polishgladiator

Something doesn't smell right. Such sloppy errors with measurement and comparison (from people who are supposedly experts?), and cageyness about answering technical questions, remi…

Actually no -- that post shows they are not performing measurements and comparisons correctly. These are not serious people.

OK, so this is a case of bad measurement and comparison. If you bothered to look at the llama.cpp output, you would see this line: llama_model_load_internal: offloaded 32/35 layers…

> [...] llama.cpp is a fantastic framework to run models locally for the single-user case (batch=1) > [...] I don't think it would be particularly fair to compare and show that MKM…

> If anyone has specific technical questions I'd be happy to answer as best I can. What is the context size for these measurements? Is it the full 4k for llama-2? And just to be cl…

Based on the integration examples, I don't think they are simply repackaging llama.cpp Rather it looks like they are reimplementing their own quantization scheme, in such a way tha…

I've been doing some hacking with Llama2 on an AMD 7900 XTX this weekend, using llama.cpp and q5_k_s quantization. Compared to MK600 on an RTX 4090 in their data, I am measuring hi…

In the realm of text editors, Vim has been my unwavering companion for 25 years. It has become an integral part of my daily existence, an inseparable bond that defies the imaginati…