Skepticism of heavy tracking is absolutely justified, so I want to address that first. I've read Richard Stallman's blog for the last 15 years, and he has had a lot of good thoughts about it. I think everyone should have access to privacy, but I don't think everything should be private.
Games are a good example. Nobody would expect a baseball player to object to tracking statistics. That's a big part of what makes the game. Online gaming is the same way. Tracking achievements adds to the fun for a lot of people.
But there are also larger social issues where tracking can be beneficial. We live in a world with a lot of diversity and an increasing amount of information. People get overwhelmed and tend to revert to tribal thinking, attacking anything that doesn't fit their group's perspective. I don't know if people on their own could ever get over this type of behavior in a world that's impossible to keep up with without taking mental shortcuts and relying on summaries of what's happening.
Personalized deep learning is an attempt to create a relatively neutral arbiter of all this information, distill it into something useful based not only on the user's behavior but also the aggregate of everyone's behavior. The algorithms don't just learn from what you like but have the potential to uncover interests and information that you might never be able to access outside your bubble.
Cortana brings that kind of aggregate information gathering to your desktop. It's an early example, and it needs lots and lots of data to learn, and the more diverse the data set it can analyze, the closer it can be to doing its job of feeding relevant information.
Windows 10 is also meant to be an Internet of Things OS. Lots of companies are working on connected devices that depend on syncing with your account. A common example for today is telling Cortana to remind you to pick up milk when you're at the store. The reminder goes to your account, and when your phone detects you're at the store, it reminds you to pick up milk.
Of course, there are people who are going to try to use this to sell you things, but that's always been the case. The hope of people working on these things is that it can bring you actually relevant suggestions instead of just the products with the largest advertising budgets. Old advertising models were very centralized and only the largest ones could really win. Personalized advertising might be able to bring the smaller but more relevant products to your attention.
Personally, I don't like advertising, and I'm not especially excited about this part of it, but that's definitely the monetary angle for it. The part that does excite me is the possibility that we can start to break down some of the communication barriers between people, get people outside of their bubbles, and bring relevant information to people based on large trends instead of isolated social groups.
There's plenty to be skeptical about here. Money tends to push things in directions that only benefit the ones with money. Microsoft and all the other IoT companies have a lot to prove before their products can be considered actually relevant for people. There's a good chance most of them will be no better than the old way of doing things. But there's a lot of potential there too.
Privacy should always be an option, but having a public online life can be good for people too.