Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Successful machine learning models: lessons learned at Booking.com

blog.acolyer.org

Post reply on HN
Print view View original

41–50 of 93 posts

Re: Successful machine learning models: lessons learned at Booking.com

#41

Post by joking »

brendanmc6 wrote: post #34
Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

That's putting it lightly. It's an abomination of a website. Truly an assault on the senses. Just booked through them yesterday, to save a few bucks-- never again. Now I understand why it's so bad-- "user interface optimization models" I refuse to believe this brings real value. The more plausible reality is, they have fantastic SEO and a tightening stranglehold on marketshare, and some AI to squeeze a few more penni…

Apart from the seo, they use a lot of money from the cut they get from the hotel to outbid the hotel on paid ads. They are more than anything a marketing agency.

Re: Successful machine learning models: lessons learned at Booking.com

#42

Post by joe_the_user »

carapace wrote: post #5

As an aside, > developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how cyberpunk that is.

>> developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness

>... I just wanted to point out how cyberpunk that is.

Nah, that is corporate flavor of the month/year/etc. It's not the 90s, so they're not "synergizing" any more but otherwise, whatever.

Re: Successful machine learning models: lessons learned at Booking.com

#43

Post by modeless »

joker3 wrote: post #9
ghaff wrote: post #3

Earlier quoted context omitted.

That was an interesting result from the original Netflix challenge. First of all it turned out that the winner wasn't actually all that useful for various reasons such as computational intensity. But, more interestingly, it also turned out that the goals of the model--"best" recommendations--isn't actually the goal of Netflix at all which is much more interested in customer retention and similar metrics. The two thin…

The big problem for Netflix was that their data was all from DVD rentals, but by the end of the contest, their business model was very streaming oriented. As you might imagine, people have beliefs about what they'll want to watch in a few days that don't exactly match up with what they want to watch now. That difference killed the model.

Thanks, this makes more sense than any other explanation I've heard for why they didn't use the Netflix prize model in production.

Re: Successful machine learning models: lessons learned at Booking.com

#44

Post by mmcclimon »

contingencies wrote: post #40
carapace wrote: post #5

As an aside, > developing an organisational capability to design, build, and deploy successful machine learned models in user-facing contexts is, in my opinion, as fundamental to an organisation’s competitiveness You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how cyberpunk that is.

I believe booking.com ran on perl for a very long time. Maybe still does. ~relevant quotes from https://github.com/globalcitizen/taoup ... In #devops is turtle all way down but at bottom is perl script. - @devops_borat Comedy: You, trying to launch a startup from scratch using Java. Tragedy: Me, trying to debug 27k lines of legacy Perl that brings $113MM/yr - @NeckbeardHacker

It still does. Booking and ZipRecruiter are easily the two largest employers of Perl programmers.

Re: Successful machine learning models: lessons learned at Booking.com

#45

Post by anthony_doan »

huac wrote: post #36
anthony_doan wrote: post #7

Experimental design is just a t-test? At least accord to that picture it seems that way. There are no ANOVA or interaction test? Do websites usually just use t-test only? Like adding one feature at a time?

say you want to try adding two features, which don't you think interact with each other, e.g. a change to the "pick this room button" and a change to the checkout flow. then, you can randomly assign users to two experiments, independently. your t-test results should then be valid if the two features are independent.

> which don't you think interact with each other,

> your t-test results should then be valid if the two features are independent.

Assuming that your assumption are correct on interaction effect.

You can do a hypothesis test on that assumption while including both factors (the two features). Which will clear away any doubt with a 95% confidence or hire a statistician =).

Re: Successful machine learning models: lessons learned at Booking.com

#46

Post by tgsovlerkhgsel »

Odenwaelder wrote: post #17

"Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings." Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

This. I still (grudgingly) use them because they also seem to have figured out that its important that your booking process works and it's friction-free, and they often do have the best price.

But if I find an alternative that has the same width of offers and a booking process that doesn't feel like a drill sergeant constantly yelling "BOOK NOW YOU WORTHLESS SCUM, BOOK, BOOK, WHAT ARE YOU WAITING FOR YOU IMBECILE, CLICK IT, BOOK, NOW, NOW" - what do they think will happen?

Re: Successful machine learning models: lessons learned at Booking.com

#47

Post by alexhutcheson »

romaaeterna wrote: post #39
orasis wrote: post #35

I’m glad they highlighted inference latency. This is a big issue that I’ve started running into at scale.

Coming from the math side, I don't really get this. Isn't all of the latency introduced by the learning side of things? Shouldn't the answer side be entirely decoupled from the learning, and simply be plugging data into an equation with a bunch of constant parameters (with values discovered by your learning system, updated at a less than realtime frequency)?

The "equation with a bunch of constant parameters" generated by an ML model can be huge, with thousands of inputs or more. Evaluating that equation for a specific observation can require a huge number of computations, which is why there's a boom in ML inference hardware right now.

Re: Successful machine learning models: lessons learned at Booking.com

#48

Post by anthony_doan »

amirmasoudabdol wrote: post #14
anthony_doan wrote: post #7

Experimental design is just a t-test? At least accord to that picture it seems that way. There are no ANOVA or interaction test? Do websites usually just use t-test only? Like adding one feature at a time?

It's even worst than that. Most of the time the validity of the t-test that they are running is questionable. They are technically running an online t-test and as soon as they find significance they stop. This is fundamentally wrong, and not conclusive at all.

Cheating on a hypothesis test... that's terrible.

Re: Successful machine learning models: lessons learned at Booking.com

#50

Post by joezydeco »

dekhn wrote: post #27
Bootwizard wrote: post #20

Earlier quoted context omitted.

What do you mean by "dark patterns"? I'm not familiar with that term

Would you like to learn more about dark patterns? YES or LATER?

47 OTHER PEOPLE ARE REPLYING TO YOUR COMMENT RIGHT NOW
Post reply on HN