Viewing profile — pongogogo

They note in the paragraph I quoted at the top that prompting has a big impact on behaviour, so yes this would work. I think that's not what METR are interested in though.

I would say this is quite a fun post and worth reading, to quote: " For our task suite, we define “cheating” as behavior where the model improves evaluation performance by exploiti…

Beautiful site, worth a read.

It's hard to tell from the data, it's so concentrated within a handful of companies who are all buying from eachother, so it feels like the contagion risk is low. At the same time …

The post mentions an approach of using a large model to generate labels and then distilling this into a smaller model to lower cost (though it doesn't provide an example)

I've been meaning to write a post like this for a while but you've done a much better job.