Viewing profile — yaodub

[dead]

Fable just came out and I wanted to see what it was actually capable of. The game: an airport security sim where manage passengers and keep them from getting angry. You place rope …

Your coworkers haven't changed. What changed is that people can hand off work they never had to think through themselves. So you don't know what they checked and you don't know wha…

[dead]

Sure, but you're paying the higher rate just to find out if that's true for your use case.

On capability, yeah. On pricing, they just told most developers they're not the target anymore.

SWE-Bench measures single tasks in isolation. In a real loop the model usually loses track of what I was trying to do long before code quality becomes the issue.

Depends whether "unable to fully automate" means "needs occasional human checkpoints" or "slowly stops caring about your actual goal." Pretty different.

[dead]

Built a quant system that reads earnings transcripts for what management is trying not to say. The model is surprisingly bad at this. Turns out management is too.

Interesting middle ground between full WASM lockdown and a bare environment. Did you end up needing to block anything else beyond the package manager?

Same with legal questions, tbh. Spots the same issues, completely disagree on which ones matter. Maybe you just need a third model to choose between the outputs lol

Have you found integrating outputs from different frontier labs consistently improves final results, or is it just kind of voodoo?

[dead]