Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Investigating how the New York Times A/B tests their headlines

blog.tjcx.me

Post reply on HN
Print view View original

81–90 of 131 posts

Re: Investigating how the New York Times A/B tests their headlines

#81

Post by unethical_ban »

systemvoltage wrote: post #79
bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

I’d agree with you but it seems like these alterations are pretty benign. They’re still professional and not at all fake/misinformation.

The entire thesis of the OP is that it is not benign and the intent is malice/inflammation rather than accurate summary of content.

It may not be "fake news" in the Fox srtain, but the OP's point is that it is not for the good of the reader or society.

Re: Investigating how the New York Times A/B tests their headlines

#82

Post by unethical_ban »

lupire wrote: post #45
throw_m239339 wrote: post #26

Earlier quoted context omitted.

> Should a news source be optimizing for engagement, or for accuracy in communicating facts? A journal is a business, with something to sell, the news, and the attention of their readers to advertisers as well. > The New York Times is a big deal. As they tell their advertisers, the NYT is the #1 news source for young, rich thought leaders: Obviously ultimately it reflects badly on the profession, since all these news…

Do young people read the NYT?

A simple answer to a simple question: Yes.

Re: Investigating how the New York Times A/B tests their headlines

#83

Post by JPKab »

clairity wrote: post #67
lwf wrote: post #63

Earlier quoted context omitted.

The Cable-Satellite Public Affairs Network[1] isn't publicly funded, no. PBS, NPR, and, via the Corporation for Public Broadcasting[2], lots of individual, independent local radio and television stations. [1]: https://en.wikipedia.org/wiki/C-SPAN [2]: https://en.wikipedia.org/wiki/Corporation_for_Public_Broadca...

npr gets like 2% of its funding from public sources, despite being national public radio . they're as beholden to moneyed and corporate interests as nytimes, and as partisan and outrage-oriented too. i stopped donating years ago.

I had to stop as well.

The article defending looting was absolutely a breaking point.

Re: Investigating how the New York Times A/B tests their headlines

#84

Post by unethical_ban »

I think there is some bad intent at times - I think a reputable institution should be optimizing for accuracy and engagement, not sensation for its own sake. Different headline styles are suitable for different sections of the paper, as well.

I saw one the other day from Nick Kristof: "America is not designed for the anatomically correct". It was about the need for public restrooms, but the headline didn't make me read it. It sounded goofy and, other than knowing Kristof writes decent articles, I passed it over.

Later, the headline was "America isn't designed for those who pee". A bit coarse, but very much more clear as to what the topic of the editorial was.

Re: Investigating how the New York Times A/B tests their headlines

#85

Post by h00kwurm »

awhitby wrote: post #30

This is interesting but unless I missed it the author doesn't really explain why they believe they observe all A/B tests. They kind of assume that the randomization is over time (so every reader within a window sees the same headline, and then it changes) rather than within cohorts at a fixed time. But the quote included suggests the NYT does do the latter: "Half of readers will see one headline, and the other half w…

you’re right! as someone who has built (and builds the future of) the testing/algorithmic machinery at the NYT, this is one of the “misses” in the post.

Re: Investigating how the New York Times A/B tests their headlines

#86

Post by johncena33 »

site-packages1 wrote: post #13
JasonFruit wrote: post #12

Earlier quoted context omitted.

Yes, but not at the cost of accuracy.

But multiple headlines can both be accurate with one being more engaging. These aren't mutually exclusive. Article about a bank robbery: "First National Bank Robbed" "Gunmen Rob First National Bank in Daring Robbery"

That doesn't mean what OP has mentioned is wrong. NYT has a history of misleading headlines [1].

[1] https://thepostmillennial.com/nyt-makes-terrorism-story-into...

Re: Investigating how the New York Times A/B tests their headlines

#87

Post by h00kwurm »

I’m really happy that folks are digging into the perceptual outcomes of the AB testing from headlines. Frankly, it provides context to the NYT’s own research and development of the algorithms and human processes that go into such efforts.

If the narrative is entirely that if we dont actively consider capturing interest, we’d be doddering and hard to track, if we do we’re abusive, then media is forever doomed to be unsatisfactory. We all hope to improve.

In the world of headlines, the “spiciness” that’s been advanced as a function of engagement hunting is something that’s currently contended with through human intervention. All headlines are human created and the outcomes of AB tests are more about improving the understanding between author/editor and captured audience than manipulation or future interest conditioning.

[disclosure: i lead ML platforms and the algo related eng products at NYT. all thoughts are my own presentation of what i have experienced, and not company opinions]

Re: Investigating how the New York Times A/B tests their headlines

#88

Post by refurb »

monch1962 wrote: post #50
JasonFruit wrote: post #6

Should a news source be optimizing for engagement, or for accuracy in communicating facts? I understand it's a business, but it's analogous to a bakery that labels all its goods as its best-selling items, only to confuse and disappoint the buyer when they open the box and find something different inside. It may sell that product that time, but it goes against the purpose of the organization as a whole, and doesn't se…

It _usually_ makes sense to optimise for engagement, particularly in the US where there's no state-funded media outlet (cue discussion about socialism that will be ignored). The UK has the BBC, Australia has the ABC, and both are state funded media outlets that aren't (at least overtly) driven by views. I'm sure their funding largely depends on how many people are consuming their product, but it's not as though some…

Based on the amount of click-bait from the BBC that gets posted to HN, I’m not sure your state funded option is much better.

I know the CBC in Canada is absolute crap - 80% of the reporting is sensationalistic and for the past 4 years has been focused on the nuances of Trump - not exactly relevant to Canada. At least not deserving of more coverage than domestic issues.

Re: Investigating how the New York Times A/B tests their headlines

#89

Post by tjs8rj »

wp381640 wrote: post #62
mxcrossr wrote: post #52

Earlier quoted context omitted.

If people don’t like click bait I recommend not using news aggregators that only show the headline and are curated based on the votes of users who don’t read the articles!

You know of an aggregator that shows the actual headlines? The ones I read use a usually poorly editorialised headline created by the user.

I made zipnews.io, aggregates the news of the day on any topic and sends you a short blurb summarizing the top 5 stories. Headlines are included.

Re: Investigating how the New York Times A/B tests their headlines

#90

Post by ipsum2 »

bArray wrote: post #76

I would suggest that this can be more 'evil' that just trying to get more clicks. A number of times I have seen on some place like Facebook where the initial article has some extreme headline, and then hours after when engagement is up, the headline is swapped to one that is less inflammatory. A few more extreme-perspective friends will send me an article saying "see?!?!" - and by the time I see it it's already been…

Yes, there's a lot of good examples.

Here's one: https://nypost.com/2020/06/02/new-york-times-changes-headlin...

Before: “As Chaos Spreads, Trump Vows to ‘End It Now"

Edited: "Trump Threatens to Send Troops into States"

Another one:

https://thehill.com/homenews/media/489013-trump-rips-ny-time...

Before: "Democrats Block Action on $1.8 Trillion Stimulus"

After: "Democrats Block Action on Stimulus Plans, Seeking Worker Protections"

And changed yet again: "Partisan Divide Threatens Deal on Rescue Bill"

https://www.techdirt.com/articles/20200413/11142144293/lessi...

Before: "A Harvard Professor Doubles Down: If You Take Epstein's Money, Do It in Secret"

After: "What Are the Ethics of Taking Tainted Funds?"

To be fair, its not only the NYTimes that does this. One that was circulating around Twitter yesterday was: "How I got COVID after taking the vaccine" with a footnote in the story: "Author did not get COVID". The headline was changed to "Why could you still get COVID-19 after receiving a vaccine?"

Post reply on HN