It’s been like that since August 2000 when they launched their premium sponsorship program, which was replaced by AdWords later that year.
I’m not sure you can fully blame Google for what the web became though. Part of the reason search engines are close to useless here 24 years later is because most of the content people produce go to SoMe sites. Sites like Hacker News, sites like Medium and so on and because of how everything became a popularity matrix people are posting soooo much useless stuff online. Where it used to be you actually had to have something to say to bother putting it online, because it took an effort, we now live in a world where 90% or the talk people do is while they’re on the shitter. Those 90% is a number I pulled out my ass, but it would be hilarious to see an actual number on just how much internet “content” gets created by people on a toilet.
It is what it is though. To make a functional search engine today which was on par with Google in 2000, you’d probably need access to browser bookmarks of the people you’ve ranked as having interesting taste in what they bookmark. Which probably isn’t really possible without breaking a lot of laws, if at all.