Viewing profile — HashedViking

Good take on that. I still think a q8 32B model with a 200k context would fit into the 48Gb VRAM of one of those modded RTX 4090.

Summarization quality depends on the question, of course mathematical or programming requests will benefit from reasoning models, but for some casual researches consisting of a few…

Hello there, I'm the coauthor of this project (UI part). I've joined this project when it was below 100 stars (a week ago), motivated by the 'local' sentiment. I think all of those…