The particularly worrying thing here is that they're now going to be gathering training data for a conversational model on _how to influence people effectively even when they already know they are being influenced_. Even more than RLFH already does. "We had to build the Torment Nexus so our children could eat" is not a good reason to build the Torment Nexus. The fact that they are not committing to not doing this tel…
While this is true, it is less effective than you might think in practice. Researchers studied something similar six months ago: https://arstechnica.com/science/2025/12/researchers-find-wha... Of course it is possible Google will create something orders of magnitude better but I doubt it. Amazon is already doing product recommendation on their front page and while it certainly drives sales it doesn’t turn people into…
It's very hard to get a huge hit without marketing - even great word of mouth benefits from amplification - but it's also near-impossible to force a hit into an audience that isn't vibing with it. The highest-grossing movies, or highest-listened pop music, is a combo of marketing + accurately hitting extremely-common/trendy tastes. See also iPhone marketing vs Windows Phone marketing. I thought Windows Phone was better; none of my friends or coworkers was convinced even after I showed them my phones. The mass media consumer may not have thought much about their tastes or tried very hard to be more adventurous, but that doesn't mean they don't like the stuff they're eating.
I think this is still in many ways bad - at the very least, it's incredibly inefficient to have a billion-dollar zero-sum "pick this one over that one" industry. But I don't think it's a deadly threat. (See also any number of "tons of money and star-power behind them" failed political candidates too...)