Viewing profile — mlgoatherder

Ah but LLM.int8 (eg. as in huggingface transformers) isn't actually int8, it's a mixed precision encoding scheme that is nominally eight bits per parameter. This means custom cuda …

I've done some experiments here with Llama 13B, in my subjective experience the original fp16 model is significantly better (particularly on coding tasks). There are a bunch of syn…

Great opportunity to pitch my recent work on a similar solution for gRPC: https://github.com/sillystack/api/blob/main/transport/transp... I'm not going to make any security claims …

I recently implemented a very similar service to service gRPC authentication mechanism for Go using EC25519 and the noise protocol framework. https://github.com/sillystack/api/blob…