I'm thinking of benchmarking different models performance playing Nethack and learning the game under different constraints.
Ask HN: Is anyone giving out tokens for benchmarking LLMs?
1–2 of 2 posts
Re: Ask HN: Is anyone giving out tokens for benchmarking LLMs?
#2LMarena, now https://arena.ai. Not sure if they have API access to automate it though.