Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Successful machine learning models: lessons learned at Booking.com

blog.acolyer.org

Post reply on HN
Print view View original

51–60 of 93 posts

Re: Successful machine learning models: lessons learned at Booking.com

#51

Post by tgsovlerkhgsel »

Bootwizard wrote: post #20
Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

What do you mean by "dark patterns"? I'm not familiar with that term

Deceiving, tricking and pressuring users into taking actions.

For example, LinkedIn having a flow that has an e-mail and password box, which will get a less attentive user to just re-enter their LinkedIn credentials. But it's actually a phishing form for your e-mail, so if your LinkedIn and e-mail password is the same, you have now "consented" to have your address book scraped and your contacts spammed.

Or, in the case of Booking.com:

* Every step has items designed to pressure you to book NOW because it'll be too late otherwise:

- "booked x times in the last x hours" on the listing, or

- "Only 1 room left!" (they now add "on our site" after they lost a consumer protection lawsuit)

- Showing booked-out listings "You missed it"

- Various notifications like "last booked X minutes ago" and "limited supply" popping in while you're scrolling to raise the pressure

* Misleading or deceptive claims

- "Jackpot, this is the cheapest price you've seen" (emphasis should be on "you've seen", this will be shown even if you look at overpriced properties)

- They seem to have stopped the "one person looking at this property" thing (to make you think that it may be gone if you don't book now - that one person is you), probably after being forced to do so by court

- a misleading rating system (the lowest possible rating is 2.5/10, and you rate category-by-category, which means that if the staff is friendly and the hotel is in a good location etc. but the rats and cockroaches ate your luggage while you slept, that's an 8/10 property - in practice, you should assume that anything below 8 is not good, below 7.5 is bad, below 7 is catastrophic, below 6 you may not survive)

- I'd also assume that they mess with the reviews in various ways, like showing mostly positive ones etc., but I haven't verified that.

Overall, I like to compare the booking experience with a drill sergeant yelling into your ear to convert (book) right now, NOW, DO IT, NOW, YOU MAGGOT! They seem to have improved significantly over previous experiences with them, probably due to a combination of me getting used to ignore the yelling, or because they realized that such a bad experience pushes customers away, or because their practices got banned one by one.

It's a shame, because other than the drill sergeant, their site is great.

Re: Successful machine learning models: lessons learned at Booking.com

#52

Post by duxup »

softwaredoug wrote: post #26
Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

Booking is obsessed with maximizing conversions, which just leads to dark patterns. One lessons from all these things is they maximize for what’s easiest to measure, not what’s most important. Conversions aren’t the end all be all, nobody wants to come back to a store with the pushy salesperson.

I'll get the phrasing wrong here but "What is easy to measure will be deemed important what is difficult to measure will be deemed unimportant."

Re: Successful machine learning models: lessons learned at Booking.com

#53

Post by joegahona »

cm2012 wrote: post #31
jwc1 wrote: post #23

Earlier quoted context omitted.

It's amazing how once you put a label on something, you start noticing it _everywhere_. Someone pointed out "confirmshaming" to me a few years ago...and since then I feel like it shows up on > 50% of the sites I visit. https://www.darkpatterns.org/types-of-dark-pattern/confirmsh...

How is confirm shaming a dark pattern? It puts more factual detail near the yes or no options so people know what they're agreeing to or declining.

> How is confirm shaming a dark pattern? It puts more factual detail near the yes or no options so people know what they're agreeing to or declining.

In its best form, perhaps. But more likely than not it looks something like this:

[ ] YES, I want to fight racism by subscribing to CrappyPublisher.net's twice-daily newsletter! [ ] NO, I am a racist (and also a pedophile)!

Re: Successful machine learning models: lessons learned at Booking.com

#54

Post by opportune »

dekhn wrote: post #27
Bootwizard wrote: post #20

Earlier quoted context omitted.

What do you mean by "dark patterns"? I'm not familiar with that term

Would you like to learn more about dark patterns? YES or LATER?

I would like you to remind me every day until I submit to your annoyance, I mean tomorrow please

Re: Successful machine learning models: lessons learned at Booking.com

#55

Post by 7thaccount »

contingencies wrote: post #40
carapace wrote: post #5

As an aside, > developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how cyberpunk that is.

I believe booking.com ran on perl for a very long time. Maybe still does. ~relevant quotes from https://github.com/globalcitizen/taoup ... In #devops is turtle all way down but at bottom is perl script. - @devops_borat Comedy: You, trying to launch a startup from scratch using Java. Tragedy: Me, trying to debug 27k lines of legacy Perl that brings $113MM/yr - @NeckbeardHacker

I read a lengthy blog post on how Booking.com basically has people code live in production (slight exaggeration) and they're fine with it, due to some monster of a monitoring test suite.

Re: Successful machine learning models: lessons learned at Booking.com

#56

Post by puranjay »

Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

I've learned to ignore the dark patterns. Still use them because their free cancellation booking process takes a lot of the pain out of picking a hotel.

Re: Successful machine learning models: lessons learned at Booking.com

#57

Post by sangnoir »

tgsovlerkhgsel wrote: post #46
Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

This. I still (grudgingly) use them because they also seem to have figured out that its important that your booking process works and it's friction-free, and they often do have the best price. But if I find an alternative that has the same width of offers and a booking process that doesn't feel like a drill sergeant constantly yelling "BOOK NOW YOU WORTHLESS SCUM, BOOK, BOOK, WHAT ARE YOU WAITING FOR YOU IMBECILE, CL…

> what do they think will happen?

Booking.com A/B tests everything: the drill-sergeant-like funnel probably has higher conversion rates than any gentler variation. So to answer your question - they think you might not book through them without the shoutiness.

Re: Successful machine learning models: lessons learned at Booking.com

#58

Post by sandGorgon »

>>Booking.com go to some lengths to minimise the latency introduced by models, including horizontally scaled distributed copies of models, a in-house developed custom linear prediction engine, favouring models with fewer parameters, batching requests, and pre-computation and/or caching.

Any idea what these are ? especially the pre-computation/caching and batching. I'm not able to see what advantage does batching bring...or how you can really cache a prediction request

Re: Successful machine learning models: lessons learned at Booking.com

#59

Post by zetazzed »

orasis wrote: post #35

I’m glad they highlighted inference latency. This is a big issue that I’ve started running into at scale.

Could you expand on that a little? Are you mostly fighting with high latency for deep learning models for imaging/audio or for traditional ML models on tabular data too? What sorts of latency SLAs do you aim for? (I do some work in this area and am always interested to hear war stories of inference issues.) Thanks!

Re: Successful machine learning models: lessons learned at Booking.com

#60

Post by psandersen »

romaaeterna wrote: post #39
orasis wrote: post #35

I’m glad they highlighted inference latency. This is a big issue that I’ve started running into at scale.

Coming from the math side, I don't really get this. Isn't all of the latency introduced by the learning side of things? Shouldn't the answer side be entirely decoupled from the learning, and simply be plugging data into an equation with a bunch of constant parameters (with values discovered by your learning system, updated at a less than realtime frequency)?

The feature engineering is probably where most of the performance comes from so there is likely a lot of code that turns the raw data into features.

Features likely include more than just the single users history, so they need to be updated often enough for the model to do fast predictions. E.g. you want your model to capture if many people are booking from the same area at once because there were results from a sports game etc, but you dont want to run an expensive query for every user of the page.

Definitely not the first thing to worry about in a startup, but better performance at Booking.com's scale is serious $$$.

Post reply on HN