Most people want a search engine designed to understand what they want in context.
But until people pay for that search capability directly, the economic incentives will not align.
And there is the ever present temptation of accepting money from advertisers. This is potentially lucrative. Will consumers ever pay enough directly to offset seller interests to sway search results? I doubt it under the current regulatory framework.
We need changes.
Some people (policy people and economists mostly) know about Ronald Coase. One of his key points is we can rebalance who has the upper hand at the beginning of a market (initial conditions) and still have economic efficiency.
Coase's classic analysis compares two hypothetical worlds. In one world, smoking is legal and non-smokers have to compensate smokers in order to have a smoke free experience. In the other world the opposite is true. Both worlds can be completely economically efficient (defined as the equilibrium where there are no additional exchanges that make all parties better off). The difference between the worlds is purely distributional --
who gets to have more money.
Distributional questions have a considerable ethical component. Like most people, I prefer the second over the first.
Regulation could also similarly apply to search engines. We could tilt the balance in favor of consumers instead of sellers (in the case of public companies, this means investors).
What am I proposing? Simply put: more discussion about such options.
I will not try specify the best legislation for all situations in this already long comment. This does not mean no significant improvements are possible nor feasible. I simply do not want to get mired in debate over only one policy option.
---
About me: I listen to good libertarian style arguments as long as they are realistic about the standard economic model's limitations and failure modes: externalities, market power, imperfect information, nonrational consumers, and so on. / I lean liberal, so I care about transition costs too. Rapid change is hard because people have limited geographic mobility, especially in the short run. I also care about observable, measurable opportunities more than only hypothetical talk about options.