Introduction to Multi-Armed Bandits (2019) • Hacker News Board

Re: Introduction to Multi-Armed Bandits (2019)

#31

Post by BOOSTERHIDROGEN » Wed, Oct 01, 2025, 2:50 PM UTC

rented_mule wrote: post #20

BOOSTERHIDROGEN wrote: post #16

Earlier quoted context omitted.

Could you expand on control theory? Is it PID?

I was just mentioning that any feedback loops have the potential downside of coupling things together in ways that make experiments (and analysis, improvements, etc.) difficult. I used control theory as an example of another type of feedback loop because we were using controllers in other parts of the system. One was roughly the proportional term of a PID controller. Another was both the proportional and integral ter…

Impressive, thanks for sharing that tidbits.

Re: Introduction to Multi-Armed Bandits (2019)

#32

Post by Iwan-Zotow » Wed, Oct 01, 2025, 2:52 PM UTC

Brystephor wrote: post #15

Hey, an area I actually work on and use in production! MAB in our case has shown improvements in our A/B metrics. My general takeaways for my experience with MAB: * we have a single optimization goal (e.g the metric to minimize or maximize). The hard part isnt defining a optimization goal. The hard part is identifying what stakeholders actually want the tradeoff between different metrics. If you have goal A and goal…

Do you have write ups how to use it in production?

Re: Introduction to Multi-Armed Bandits (2019)

#33

Post by Normal_gaussian » Wed, Oct 01, 2025, 5:51 PM UTC

3abiton wrote: post #10

Normal_gaussian wrote: post #9

Its very easy to implement toy multi-armed bandits, and even getting them into production code isn't so bad. Its the reporting, education, and adding features without breaking everything that gets very hard very quick.

> Its the reporting, education, and adding features without breaking everything that gets very hard very quick. What's challenging about that though?

mab's get misinterpreted very easily, particularly because they tend to be useful in places where the non-technical (or low stats knowledge) people are the major stakeholders. Reporting on MAB functionality and output inline with other features without overcomplicating interpretation or leading to easy misinterpretation is an often underestimated task. Then allowing iteration on this without breaking any of the careful presentation or decisions made is a challenge for any MoC process.

Re: Introduction to Multi-Armed Bandits (2019)

#34

Post by Brystephor » Wed, Oct 01, 2025, 7:06 PM UTC

Iwan-Zotow wrote: post #32

Brystephor wrote: post #15

Hey, an area I actually work on and use in production! MAB in our case has shown improvements in our A/B metrics. My general takeaways for my experience with MAB: * we have a single optimization goal (e.g the metric to minimize or maximize). The hard part isnt defining a optimization goal. The hard part is identifying what stakeholders actually want the tradeoff between different metrics. If you have goal A and goal…

Do you have write ups how to use it in production?

No public write ups. What information would you want to see? I think there's two categories: infra ops info, and then product insights and "gotchas"/unintuitive but valid results.