[flagged]
Show HN: Optimize and serve models with Fable quality at half the cost
31–34 of 34 posts
Re: Show HN: Optimize and serve models with Fable quality at half the cost
#32Probably a silly question, but ...: if it's optimizing itself purely inside a simulator, what keeps it from just exploiting the simulator's glitches to look good on paper, while not being so good when it hits the real world?
Great question. You need the simulator to be realistic enough that the optimizer cannot reward hack, but it does not need to be perfectly realistic.
Re: Show HN: Optimize and serve models with Fable quality at half the cost
#33Probably a silly question, but ...: if it's optimizing itself purely inside a simulator, what keeps it from just exploiting the simulator's glitches to look good on paper, while not being so good when it hits the real world?
[deleted]
Re: Show HN: Optimize and serve models with Fable quality at half the cost
#34Probably a silly question, but ...: if it's optimizing itself purely inside a simulator, what keeps it from just exploiting the simulator's glitches to look good on paper, while not being so good when it hits the real world?
Great question. You need the simulator to be realistic enough that the optimizer cannot reward hack, but it does not need to be perfectly realistic.
^this
There's two ways to functionally measure this, reconstruction fidelity (which we're able to get to 0.7 - 0.95), and downstream performance (which agrees on the best and worst models, but can sometimes disagrees on the middle of the pack routing).