Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Successful machine learning models: lessons learned at Booking.com

blog.acolyer.org

Post reply on HN
Print view View original

31–40 of 93 posts

Re: Successful machine learning models: lessons learned at Booking.com

#31

Post by cm2012 »

jwc1 wrote: post #23
kennethologist wrote: post #21

Earlier quoted context omitted.

Dark Patterns are tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something. See more examples here: https://www.darkpatterns.org/

It's amazing how once you put a label on something, you start noticing it _everywhere_. Someone pointed out "confirmshaming" to me a few years ago...and since then I feel like it shows up on > 50% of the sites I visit. https://www.darkpatterns.org/types-of-dark-pattern/confirmsh...

How is confirm shaming a dark pattern? It puts more factual detail near the yes or no options so people know what they're agreeing to or declining.

Re: Successful machine learning models: lessons learned at Booking.com

#32

Post by jhncls »

Bootwizard wrote: post #20
Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

What do you mean by "dark patterns"? I'm not familiar with that term

Ben Edelman (Harvard, Microsoft) published a study [0] about how dark patterns in the online travel industry help them reach margins up to 25%. He also mentions the consolidation where most well-known booking sites are owned by just two large groups.

[0] http://www.benedelman.org/impact-of-ota-bias-and-consolidati...

Re: Successful machine learning models: lessons learned at Booking.com

#33

Post by lbotos »

cm2012 wrote: post #31
jwc1 wrote: post #23

Earlier quoted context omitted.

It's amazing how once you put a label on something, you start noticing it _everywhere_. Someone pointed out "confirmshaming" to me a few years ago...and since then I feel like it shows up on > 50% of the sites I visit. https://www.darkpatterns.org/types-of-dark-pattern/confirmsh...

How is confirm shaming a dark pattern? It puts more factual detail near the yes or no options so people know what they're agreeing to or declining.

It's emotional manipulation. Confirm shaming uses fear.

If I'm cancelling amazon prime because it "costs too much" but you say "are you sure you want to miss out on all the fast shipping" someone who is easily manipulated may continue to subscribe because they are weak willed.

You could instead ask: "Why are you cancelling?":

- Cost

- Don't use it enough

- ...

- Other (please specify:)

Re: Successful machine learning models: lessons learned at Booking.com

#34

Post by brendanmc6 »

Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

That's putting it lightly. It's an abomination of a website. Truly an assault on the senses. Just booked through them yesterday, to save a few bucks-- never again.

Now I understand why it's so bad-- "user interface optimization models"

I refuse to believe this brings real value. The more plausible reality is, they have fantastic SEO and a tightening stranglehold on marketshare, and some AI to squeeze a few more pennies out along the way. Whatever metrics they are seeing, it won't be worth it in the long run. This kind of UX and product won't last.

Re: Successful machine learning models: lessons learned at Booking.com

#36

Post by huac »

anthony_doan wrote: post #7

Experimental design is just a t-test? At least accord to that picture it seems that way. There are no ANOVA or interaction test? Do websites usually just use t-test only? Like adding one feature at a time?

say you want to try adding two features, which don't you think interact with each other, e.g. a change to the "pick this room button" and a change to the checkout flow. then, you can randomly assign users to two experiments, independently. your t-test results should then be valid if the two features are independent.

Re: Successful machine learning models: lessons learned at Booking.com

#37

Post by tixocloud »

benmaraschino wrote: post #19
jj12345 wrote: post #10

Earlier quoted context omitted.

Not sure if you'd be willing, but I'd love a quick rundown of the high level takeaways if you'd be willing to drop them here. Are you talking about more than one experimental design in terms of comparing the exp/control distributions or something else?

Not the OP, but I work on similar problems, albeit in a different setting (healthcare, millions+ of patients). The gist is that you have to bake experimental design into the deployment of your ML model, but in many cases a simple RCT or A/B test just won't cut it. This is largely because when you deploy a model, no matter how sophisticated or accurate, there's no guarantee that it'll actually move the needle in terms…

Would be really keen to hear you speak about the subject in greater detail actually. Love having to balance the practicalities of implementing a model in production and validating the outcome while not missing out on utility.

Re: Successful machine learning models: lessons learned at Booking.com

#38

Post by djp518 »

> Once deployed, beyond the immediate business benefit they often go on to become a foundation for further product development.

This is one of the reasons I am a big believer in having a system to track model research and deployment lineage. (I personally use Domino Data Lab for this. I also work for Domino, but use it in my own modeling work and that of others I mentor.) No matter which system you use to track lineage, I've found it important to have a strict history of retraining, versioning, and experimentation. When models are used in downstream systems from the one they were originally intended, it becomes even more critical to able to explain and reproduce the 'research' that led up to deployment.

Re: Successful machine learning models: lessons learned at Booking.com

#39

Post by romaaeterna »

orasis wrote: post #35

I’m glad they highlighted inference latency. This is a big issue that I’ve started running into at scale.

Coming from the math side, I don't really get this. Isn't all of the latency introduced by the learning side of things? Shouldn't the answer side be entirely decoupled from the learning, and simply be plugging data into an equation with a bunch of constant parameters (with values discovered by your learning system, updated at a less than realtime frequency)?

Re: Successful machine learning models: lessons learned at Booking.com

#40

Post by contingencies »

carapace wrote: post #5

As an aside, > developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how cyberpunk that is.

I believe booking.com ran on perl for a very long time. Maybe still does. ~relevant quotes from https://github.com/globalcitizen/taoup ...

In #devops is turtle all way down but at bottom is perl script. - @devops_borat

Comedy: You, trying to launch a startup from scratch using Java. Tragedy: Me, trying to debug 27k lines of legacy Perl that brings $113MM/yr - @NeckbeardHacker

Post reply on HN