Test machine learning the right way: regression testing
1–4 of 4 posts
Re: Test machine learning the right way: regression testing
#2Would love to hear what other testing techniques people use for machine learning? Are there great testing frameworks out there that people use?
Re: Test machine learning the right way: regression testing
#3“The latter being when the training or test data follows a different distribution to the in-operation data”
This form of ML bug is the most challenging to catch. The true in-operation distribution is often unknown which makes testing for such bugs a very challenging problem. Any thoughts on this?
Re: Test machine learning the right way: regression testing
#4“The latter being when the training or test data follows a different distribution to the in-operation data” This form of ML bug is the most challenging to catch. The true in-operation distribution is often unknown which makes testing for such bugs a very challenging problem. Any thoughts on this?
Thanks for your comment. The whole field of run-time monitoring is concerned with this problem. It's a tough one to crack when the distribution changes are subtle, but you can and should at least check simple data attributes for consistency.