Live data from Hacker News

Investigating how the New York Times A/B tests their headlines

blog.tjcx.me

41–50 of 131 posts

Re: Investigating how the New York Times A/B tests their headlines

#41
post #20

Earlier quoted context omitted.

Headlines have become advertisements. They're almost never written by the journalists who wrote the story, they're written for space constraints that change on a whim ... They're very rarely "accurate" the way stories are intended to be for these reasons.

Have journalists ever controlled or written their own headlines? I don't think this is a change from any point in newspaper history.

Why has it been historically separated like this?

Re: Investigating how the New York Times A/B tests their headlines

#42

Good piece, but damn I was hoping for some details on their implementation. Are the variations coming from the server, CDN/edge, or client? How does this impact SEO? Do they change the article slug if the title is a winner? How much is automated and how much is human curated? When beset with new assignments who has the time to babysit these things?

cdn; seo impact is also tested for; canonical URL doesn't change; titles are written by humans, testing is entirely automated; small parts of larger UX team just do testing so babysitting these is their job

Re: Investigating how the New York Times A/B tests their headlines

#43

Earlier quoted context omitted.

Have journalists ever controlled or written their own headlines? I don't think this is a change from any point in newspaper history.

Why has it been historically separated like this?

Because the headline isn't part of the story, it's part of the index/ToC of the whole paper, so it's written by the staff that put the whole paper together.

Re: Investigating how the New York Times A/B tests their headlines

#44

Earlier quoted context omitted.

Yes, but not at the cost of accuracy.

But multiple headlines can both be accurate with one being more engaging. These aren't mutually exclusive. Article about a bank robbery: "First National Bank Robbed" "Gunmen Rob First National Bank in Daring Robbery"

"Engaging" while being highly irritating. The first one is concise.

Re: Investigating how the New York Times A/B tests their headlines

#45

Should a news source be optimizing for engagement, or for accuracy in communicating facts? I understand it's a business, but it's analogous to a bakery that labels all its goods as its best-selling items, only to confuse and disappoint the buyer when they open the box and find something different inside. It may sell that product that time, but it goes against the purpose of the organization as a whole, and doesn't se…

> Should a news source be optimizing for engagement, or for accuracy in communicating facts? A journal is a business, with something to sell, the news, and the attention of their readers to advertisers as well. > The New York Times is a big deal. As they tell their advertisers, the NYT is the #1 news source for young, rich thought leaders: Obviously ultimately it reflects badly on the profession, since all these news…

Do young people read the NYT?

Re: Investigating how the New York Times A/B tests their headlines

#46

The article is trying to paint NYT's A/B testing as nefarious, but unfortunately in the current social-media-oriented landscape, headlines are very, very important for clarity, and A/B testing headlines is something every media publisher with a sufficient tech infra does. As frequent Hacker News submitters know, headlines alone can determine whether a post gets upvoted.

> but unfortunately in the current social-media-oriented landscape, headlines are very, very important for clarity,

Clarity? Or click-thrus? It's definitely not the former. Which in turn discounts the NYT as a beacon for journalisms and associated standards. They're desperate for traffic. Clarity isn't a metric in the playbook.

Witin the world of high volume publishing, this type of A/B testing was popularized BuzzFeed.

Re: Investigating how the New York Times A/B tests their headlines

#47

Good piece, but damn I was hoping for some details on their implementation. Are the variations coming from the server, CDN/edge, or client? How does this impact SEO? Do they change the article slug if the title is a winner? How much is automated and how much is human curated? When beset with new assignments who has the time to babysit these things?

cdn; seo impact is also tested for; canonical URL doesn't change; titles are written by humans, testing is entirely automated; small parts of larger UX team just do testing so babysitting these is their job

+++ thank you! A stack to aspire to!

Re: Investigating how the New York Times A/B tests their headlines

#48
post #14

Earlier quoted context omitted.

Is there actually an either/or here? Surely it’s possible to write two (or more) headlines for a story that are all equally accurate?

Are you saying there isn't just two sides to every story?

Yes, so A/B testing won't work. You need a one-armed bandit algorit.

Re: Investigating how the New York Times A/B tests their headlines

#49

Valuable anecdata. News is defined by conflict, (literally, no conflict = not news) and this shows how headlines that express the most conflict get the most clicks. The integrity issues come up when even though the facts reported are real, the conflict that frames them is manufactured - and this is why people reject news. They don't reject it because of fake facts, they reject it because of fake conflict. When I want…

Meh. This is the narrative of Marx and Foucault, and one I think is demonstrably false. For multiple counter-examples, see pretty much everything on https://www.msn.com/en-us/news/good-news

Every single one of those headlines asserts a conflict, and the only difference is these MSN ones appeal to sentiment instead of outrage.

Arguably, newspapers as a concern are a post-Marx phenomenon, and it's what people get taught in j-school, and what editors accept. Students of both are peddlers of pernicious nonsense, but their students also produced some entertaining and powerful things, notably recycled recipes for accelerating and reverting democracies into their inevitable tyrannies.

Anyway, the NYTimes doesn't register as meaningful to me, and if they were looking for a reason why people are leaving them and other mainstream outlets behind, it's becuase their conflicts are contrived.

Re: Investigating how the New York Times A/B tests their headlines

#50

Should a news source be optimizing for engagement, or for accuracy in communicating facts? I understand it's a business, but it's analogous to a bakery that labels all its goods as its best-selling items, only to confuse and disappoint the buyer when they open the box and find something different inside. It may sell that product that time, but it goes against the purpose of the organization as a whole, and doesn't se…

It _usually_ makes sense to optimise for engagement, particularly in the US where there's no state-funded media outlet (cue discussion about socialism that will be ignored). The UK has the BBC, Australia has the ABC, and both are state funded media outlets that aren't (at least overtly) driven by views. I'm sure their funding largely depends on how many people are consuming their product, but it's not as though some editor is going to be fired that afternoon if their "Oprah/Meghan" story loses out to the Murdoch equivalent. I'd be curious to compare how A/B testing works for state-funded media vs pay-for-access media outlets
Post reply on HN