Earlier quoted context omitted.
The premise here is that the AI is operating at the behest of someone else. Now suppose that it's open source and runs on your own device. "It will learn when your ADHD medication typically starts wearing off and you're more prone to act impulsively" and then avoid presenting you with purchasing decisions at those times etc. It will digest both ad-laden blogspam and human psych research papers so it can present the i…
We've been trying to push FOSS software into peoples lives for many decades now and they still mostly opt for stuff from the big tech. It is very likely that the same will happen again with AI unfortunately.
"People don't care" is at best defeatism and at worst explicit propaganda designed to discourage people from making the attempt. Sometimes it works, sometimes it doesn't. It works more often when you try than when you don't.