Live data from Hacker News

Successful machine learning models: lessons learned at Booking.com

blog.acolyer.org

21–30 of 93 posts

Re: Successful machine learning models: lessons learned at Booking.com

#21

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

What do you mean by "dark patterns"? I'm not familiar with that term

Dark Patterns are tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something.

See more examples here: https://www.darkpatterns.org/

Re: Successful machine learning models: lessons learned at Booking.com

#22
post #10

Earlier quoted context omitted.

Not sure if you'd be willing, but I'd love a quick rundown of the high level takeaways if you'd be willing to drop them here. Are you talking about more than one experimental design in terms of comparing the exp/control distributions or something else?

Not the OP, but I work on similar problems, albeit in a different setting (healthcare, millions+ of patients). The gist is that you have to bake experimental design into the deployment of your ML model, but in many cases a simple RCT or A/B test just won't cut it. This is largely because when you deploy a model, no matter how sophisticated or accurate, there's no guarantee that it'll actually move the needle in terms…

Do you know a good introduction to adaptive/sequential designs? I'm looking for something along the lines of a textbook aimed at a graduate level seminar.

Re: Successful machine learning models: lessons learned at Booking.com

#23

Earlier quoted context omitted.

What do you mean by "dark patterns"? I'm not familiar with that term

Dark Patterns are tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something. See more examples here: https://www.darkpatterns.org/

It's amazing how once you put a label on something, you start noticing it _everywhere_.

Someone pointed out "confirmshaming" to me a few years ago...and since then I feel like it shows up on > 50% of the sites I visit.

https://www.darkpatterns.org/types-of-dark-pattern/confirmsh...

Re: Successful machine learning models: lessons learned at Booking.com

#24

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

I think both of those statements are true. They do try to manipulate their users, and the listings you see on Booking.com have a lot of content too!

Re: Successful machine learning models: lessons learned at Booking.com

#25

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

What do you mean by "dark patterns"? I'm not familiar with that term

[deleted]

Re: Successful machine learning models: lessons learned at Booking.com

#26

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

Booking is obsessed with maximizing conversions, which just leads to dark patterns.

One lessons from all these things is they maximize for what’s easiest to measure, not what’s most important. Conversions aren’t the end all be all, nobody wants to come back to a store with the pushy salesperson.

Re: Successful machine learning models: lessons learned at Booking.com

#27

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

What do you mean by "dark patterns"? I'm not familiar with that term

Would you like to learn more about dark patterns? YES or LATER?

Re: Successful machine learning models: lessons learned at Booking.com

#28
post #5

As an aside, > developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how cyberpunk that is.

That gave me a flashback to the early 2000s AIML chatbot craze. "What, you have an online store and no customer chatbot? What are you doing ?!"

Flipside is that this ML (whether you consider it AI or not) really is delivering huge value to the businesses that deploy them.

Re: Successful machine learning models: lessons learned at Booking.com

#29

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

Booking is obsessed with maximizing conversions, which just leads to dark patterns. One lessons from all these things is they maximize for what’s easiest to measure, not what’s most important. Conversions aren’t the end all be all, nobody wants to come back to a store with the pushy salesperson.

> maximize for what’s easiest to measure

Just like the unholy abomination that professional project managers have turned “agile methodologies” into.

Re: Successful machine learning models: lessons learned at Booking.com

#30

Experimental design is just a t-test? At least accord to that picture it seems that way. There are no ANOVA or interaction test? Do websites usually just use t-test only? Like adding one feature at a time?

It's even worst than that. Most of the time the validity of the t-test that they are running is questionable. They are technically running an online t-test and as soon as they find significance they stop. This is fundamentally wrong, and not conclusive at all.

A few years ago when I was still working there, involved in the experimentation tooling among other things, we largely excised that particular behavior. What are you basing your assertion on?
Post reply on HN