Viewing profile — 3d27

Checkout this instead: https://github.com/confident-ai/deepeval Also has native ragas implementation but supports all models.

This is great. I'm also building an LLM evaluation framework with all these benchmarks integrated in one place so anyone can go benchmark these new models on their local setup in u…

(found this interesting post on medium, this is not my original work)

There's a lot more in the evaluation space, including this one: https://github.com/confident-ai/deepeval

I'm just imagining Jensen Huang laughing in his sleep right now...

How did you calculate accuracy and bias?

The package I built is like a provider for 10+ different evaluation metrics that run both locally on your machine using models from hugging-face but also on the cloud IF you want m…