Viewing profile — jojo2219

what's the state of the art in quantization methods these days that one might apply to a model like LLama 3? Any particular literature to read? Of course priorities differ across m…

weights and activations all bf16?

Where are f32 and f16 used? I see a lot of `.float()' and `.type_as()' in the model file, and nothing explicit about f16. Are the weights and all the activations in f32?