Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Investigating how the New York Times A/B tests their headlines

blog.tjcx.me

Post reply on HN
Print view View original

91–100 of 131 posts

Re: Investigating how the New York Times A/B tests their headlines

#91

Post by drak0n1c »

ipsum2 wrote: post #90
bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

Yes, there's a lot of good examples. Here's one: https://nypost.com/2020/06/02/new-york-times-changes-headlin... Before: “As Chaos Spreads, Trump Vows to ‘End It Now" Edited: "Trump Threatens to Send Troops into States" Another one: https://thehill.com/homenews/media/489013-trump-rips-ny-time... Before: "Democrats Block Action on $1.8 Trillion Stimulus" After: "Democrats Block Action on Stimulus Plans, Seeking Worker…

NewsDiffs is a scraper that did this kind of automatic headline change tracking, but it hasn't been updated in a while.

https://github.com/ecprice/newsdiffs

This alternative looks more recent

https://github.com/DocNow/diffengine

Re: Investigating how the New York Times A/B tests their headlines

#92

Post by drak0n1c »

ftio wrote: post #3

If you're interested in observing this in realtime, there's a great Twitter account called @nyt_diff[0] that automatically posts when headlines on the front page change. Really cool to watch articles get updated as stories evolve. It's particularly interesting to witness headline writers grapple with how and how much to editorialize. [0] https://twitter.com/nyt_diff

There's a project inspired by nyt diff and NewsDiffs and working on improving the algorithm and supporting many more news outlets:

https://github.com/DocNow/diffengine

Re: Investigating how the New York Times A/B tests their headlines

#93

Post by CyanBird »

drak0n1c wrote: post #91
ipsum2 wrote: post #90

Earlier quoted context omitted.

Yes, there's a lot of good examples. Here's one: https://nypost.com/2020/06/02/new-york-times-changes-headlin... Before: “As Chaos Spreads, Trump Vows to ‘End It Now" Edited: "Trump Threatens to Send Troops into States" Another one: https://thehill.com/homenews/media/489013-trump-rips-ny-time... Before: "Democrats Block Action on $1.8 Trillion Stimulus" After: "Democrats Block Action on Stimulus Plans, Seeking Worker…

NewsDiffs is a scraper that did this kind of automatic headline change tracking, but it hasn't been updated in a while. https://github.com/ecprice/newsdiffs This alternative looks more recent https://github.com/DocNow/diffengine

The "Editing TheGrayLady" twitter account is the updated version that is still working

https://twitter.com/nyt_diff

Re: Investigating how the New York Times A/B tests their headlines

#94

Post by godmode2019 »

hkai wrote: post #71
CarVac wrote: post #69

Earlier quoted context omitted.

Sounds like someone needs to make an aggregator that shows only brief auto-generated summaries, no headlines or images.

Even better, these autogenerated summaries should give most weight to paragraphs at the end of the article. I often find that New York Times and other media put important details that sometimes contradict the headline or challenge the main idea of the article somewhere close to the end of the article. This lets them still claim objectivity while preserving a highly polarizing message.

This has a name its called burying the lead, its used because news papers are structured in a way where they know people have limited attention so usually they don't read to the end. All the important bits are typically laied out linearly, often trailing off at the end.

Burying the lead takes advance if this and puts the important but inconvenient information at the bottom.

Re: Investigating how the New York Times A/B tests their headlines

#95

Post by comex »

afavour wrote: post #27
JasonFruit wrote: post #16

Earlier quoted context omitted.

Yes, but did you read the examples in the article? Many of the high-engagement headlines ignored most of the content in favor of a tangential but titillating detail. That's National Enquirer stuff, not what the New York Times has prided itself on being.

I did, but honestly most of the examples confused me. Both the Tesla rocket example and Cuomo story are surely headline changes relating to the story changing? They didn’t add “Cuomo apologizes” to titillate, it’s because… he apologized, and he hadn’t when the story was first published. Which isn’t even A/B testing! The article also more or less states this, hence my confusion.

I think the same is true for the Trump CPAC speech article. The blog post compares the initial headlines about him “claiming leadership” to the ones about him attacking other Republicans. But just from eyeballing the chart, first the headline changes from “Plans to Claim” to “Claims”, presumably around when he started giving the speech, and then shortly afterward it’s changed to mention him attacking Republicans, probably at the point in the speech where he did that. There does seem to be an A/B test between the last two headlines, “Trump Pledges G.O.P. Unity, Then Attacks Republicans Who Backed Impeachment” vs. “Trump’s Republican Hit List at CPAC Is a Warning Shot to His Party”. But that’s not the comparison the blog post makes. And among those two, while the winning headline (the second one) is clickbaitier, both focus on the same part of the speech, and neither is inaccurate.

Re: Investigating how the New York Times A/B tests their headlines

#96

Post by fwip »

xwolfi wrote: post #80
bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

Having worked in A/B testing in a loan marketing company, where we were pushing 300% payday loans in Mexico (you read that right), I can talk of evil. I think, when you're in the day to day of a web platform, you really just simplified everything from your hiring to your board meeting to one metric: click. Everything is number of click, it becomes no more evil than a Lion in the Savanna, you just must increase your n…

Evil done by beauracracy, process, or algorithm, is still evil.

Re: Investigating how the New York Times A/B tests their headlines

#97

Post by clairity »

JPKab wrote: post #83
clairity wrote: post #67

Earlier quoted context omitted.

npr gets like 2% of its funding from public sources, despite being national public radio . they're as beholden to moneyed and corporate interests as nytimes, and as partisan and outrage-oriented too. i stopped donating years ago.

I had to stop as well. The article defending looting was absolutely a breaking point.

not sure which article was your breaking point, but for me, it's been a steady downhill slide for at least 5 years, with the complete whiff on the run-up to the 2016 election being a real wake-up-and-take-stock affair.

Re: Investigating how the New York Times A/B tests their headlines

#98

Post by refenestrator »

systemvoltage wrote: post #79
bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

I’d agree with you but it seems like these alterations are pretty benign. They’re still professional and not at all fake/misinformation.

If they consistently go from "max outrage to grab attention" to "more clinical, for posterity", that's not benign. It's dishonest. Either be a tabloid or don't be.

Re: Investigating how the New York Times A/B tests their headlines

#99

Post by infogulch »

bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

Some comments on newly-released YouTube videos will restate the title of the video at the time the comment was made simply to annotate that fact for future viewers, and such comments are often highly upvoted.

An interesting reactionary adaptation to this "temporary sensational title" exploit.

Re: Investigating how the New York Times A/B tests their headlines

#100

Post by mxcrossr »

CarVac wrote: post #69
mxcrossr wrote: post #52

Earlier quoted context omitted.

If people don’t like click bait I recommend not using news aggregators that only show the headline and are curated based on the votes of users who don’t read the articles!

Sounds like someone needs to make an aggregator that shows only brief auto-generated summaries, no headlines or images.

I think user written, three sentence summaries for hacker news posts would be nice. I like that I can submit random interesting articles here, but sometimes a bit of context is necessary to get others engaged.
Post reply on HN