Viewing profile — mysterhawk

[dead]

Location: Italy Remote: Yes Willing to relocate: No Technologies: Rust,Python,Java-spring,TypeScript,React,Html,CSS,Nim,C,C++,PHP Resumé/CV: https://drive.google.com/file/d/1F_Fp3p…

With this model: https://huggingface.co/bartowski/Qwen2.5-72B-Instruct-GGUF Man I need to test the q8 version with llamafiles optimizations, it would be so nice to host it locally …

https://justine.lol/matmul/ https://www.youtube.com/watch?v=-mRi-B3t6fA Checkout these articles

I could run at 1token/s on qwen 2.5 72b q4_k_m on my i7 8750h + alot of ram XD

And everything runs way faster on cpu like that

You can already do it with llamafile, checkout the project, it lets you convert a .gguf model in a portable executable