Live data from Hacker News

The internet is an SEO landfill

docs.sendwithses.com

231–240 of 438 posts

Re: The internet is an SEO landfill

#231

Google any recipe, and there are at least 5 paragraphs (usually a lot more) of copy that no one will ever read, and isn't even meant for human consumption. Google "How to learn x", and you'll usually get copy written by people who know nothing about the subject, and maybe browsed Amazon for 30 minutes as research. Real, useful results that used to be the norm for Google are becoming more and more rare as time goes by…

Re: recipes, one of the potential reasons for this is that bare recipes are not copyrightable by law. Courts see them as algorithms, almost -- e.g., there are only so many ways to cook a grilled cheese sandwich. > 17 U.S.C. 102(b): In no case does copyright protection for an original work of authorship extend to any idea, procedure, process, system, method of operation, concept, principle, or discovery, regardless of…

oh please. the text is not there for copyright reasons. it’s there to satisfy the google bot so the article ranks higher. you’re going completely off base by thinking this is a copyright thing. nobody is thinking that far.

Re: The internet is an SEO landfill

#232
post #18

Search results are so heavily weighted toward commerce, products and services because those are the people that can spend money on SEO. It's made the internet, seen through the eyes of search results, seem like an aggressive, shady market bazaar. I try searching for anything remotely bike related, bike community, mechanical information, or just general cool bike stuff and I can't find the human community underneath a…

Pretty much captures it. That was the founding principle of Blekko which was that humans could curate a core set of 'good' websites for a topic and all the crap would not have a foot hold to show up on the page.

So to share some of the challenges with that (if anyone out there wants to try again) they are as follows:

* 'Search' means different things to different people, and we've been trained that a 'search engine' finds anything on the web (for the most part). The product Blekko built could more accurately be called a 'reference' engine which was used successfully by people trying to find facts or data and were not generally trying to find things to buy.

* Have your own advertising system, all in house, where you don't have to "revenue share" with anyone if you don't want to. At its peak Blekko was serving over 10M queries per day which, if we had owned all the advertising revenue on those searches would have kept us going and growing. That said, building an advertising system is both difficult and fraught with patent risk / bad-actor risk.

* Don't let anonymous users use the service. This is perhaps the hardest thing, most people won't give up an email address for even the most useful of services, however since you're spending money serving up search queries you don't want to waste that money serving up queries to bots and other bad actors. At any given time when I went through the logs there were between 5% up to nearly 18% of the queries were 'suspicious' or likely bots. That is 18% of your capacity you can't give to "real" humans if you can't control that traffic.

* Build a relevancy ranking system rather than a popularity ranking system. Search results have two metrics of interest, precision and recall. For a reference engine you want to focus on precision over recall. And while existing search engines use the "virtuous cycle" of search & click to track popularity (which can be an indicator of precision but is better at indicating click-baityness) build your ranking engine using NLP based evaluations.

* Your document index size should target 5 billion documents with a goal of 10 billion documents. Scale your cluster and algorithms to process a query to that index in 100mS or less.

Do that and win :-) Or find the next barrier to creating a useful way to search the web for information.

Re: The internet is an SEO landfill

#233
post #18

Search results are so heavily weighted toward commerce, products and services because those are the people that can spend money on SEO. It's made the internet, seen through the eyes of search results, seem like an aggressive, shady market bazaar. I try searching for anything remotely bike related, bike community, mechanical information, or just general cool bike stuff and I can't find the human community underneath a…

I wonder if there's a ripe market for a search engine that specializes on "clean" search results. Google grew into its current titan status by, in part, providing the results with higher relevancy and accuracy than the competition, and avoiding the primitive SEO cheats of those days (like spamming the page with irrelevant keywords to capture traffic). Maybe that's the time for an engine that could avoid modern, more sophisticated SEO cheats.

Re: The internet is an SEO landfill

#234
post #18

Search results are so heavily weighted toward commerce, products and services because those are the people that can spend money on SEO. It's made the internet, seen through the eyes of search results, seem like an aggressive, shady market bazaar. I try searching for anything remotely bike related, bike community, mechanical information, or just general cool bike stuff and I can't find the human community underneath a…

And it has resulted in communities like the ones you mention moving to aggregators like Reddit, Facebook, or even HN. And I’m ok with that - it’s like how communities IRL gather in churches, bars, or other areas.

Why shouldn’t the internet imitate actual life?

Re: The internet is an SEO landfill

#235

Earlier quoted context omitted.

Don't forget the "showing results for Y, click here to search for X" when X is what you actually wanted to search for. It's ironic that, in trying to make search "human friendly", Google has also succeeded in giving it all the negative traits of a human --- the "human" that is more like a commissioned salesperson. I know the argument is that humans frequently make mistakes so "we should just show them what we think t…

Grep the internet and sort by pagerank was great. Then everyone removed links that went offsite. Then link farms. The fake blogs and influencers are the evolution of link farms. Unfortunately sellers will game the system. How do we create a way for buyers to game it?

Pagerank could be gamed too. Buy expired domain with high pagerank, establish a forwarder to your domain and your domain inherited the pagerank.

Re: The internet is an SEO landfill

#236
Indeed most of the content on the web is SEO fluff or veiled attempts to push an affiliate link.

Worse, almost every site has a cookie “preference” popup (when everyone knows the preference is just the minimum necessary cookies), a newsletter signup popup, and a browser notification request popup.

Add to that the autoplaying videos that will pop out into their own overlay if you scroll away plus the rotating ads of different sizes which cause the page content to shift up and down... oh and the fancy morphing page headers that hang down over the content you’re trying to read if you don’t scroll enough at first.

The web is currently a shitshow of comic proportions, the likes of which not even the most cynical comedians accurately predicted.

The desktop app situation isn’t as bad, but it follows a similar trend of demanding more from the hardware and user (and network) while providing less.

It might actually be better overall if mobile and desktop performance had not increased in the last decade. I swear the end user experience hasn’t on average improved even a fraction as much as the hardware capability has.

What a dismal road we are on.

Re: The internet is an SEO landfill

#237

I append "reddit" to any query not meant for a productive task. Except now the internet is mostly reddit for me, which kinda bums me out. Especially given the new js-heavy design. In 2019, if I want reliable search about something I know nothing about I must first search on google, then try again with "reddit" appended, then change www.reddit.com to old.reddit.com, then parse through what are often questionable answe…

Am I the only one who thinks Reddit really sucks? It's a cesspool of kiddie memes, shock, ill informed opinion and political posturing. Besides, if I wanted to search Reddit I'd go to Reddit. It being front page for _everything_ I search for now is so annoying.

Browsing reddit is as described, I never go to r/all for that reason, not do I feel obliged to login, comment or vote in most cases but if you limit your consumption of reddit to a few subs, the difference is huge.

Re: The internet is an SEO landfill

#238

Earlier quoted context omitted.

Don't forget the "showing results for Y, click here to search for X" when X is what you actually wanted to search for. It's ironic that, in trying to make search "human friendly", Google has also succeeded in giving it all the negative traits of a human --- the "human" that is more like a commissioned salesperson. I know the argument is that humans frequently make mistakes so "we should just show them what we think t…

I remember the days of hiding text in the same color font as the background so that grepping would make your page the top result. So I’m not sure those days were better. The dumber search is, the easier it is for the bad actors to game it and drown out the legit sources. Maybe page rank (mentioned in a sibling response) helped address this, but then it’s really no longer just a dumb grep.

I remember the days of hiding text in the same color font as the background so that grepping would make your page the top result.

A simple "-site:somesite.com" removes those, or just scroll past them. Keyword spam is easy to ignore because the title of the page, the domain, or the path often has nothing to do with what you're searching for, and the preview text is nonsensical.

The dumber search is, the easier it is for the bad actors to game it and drown out the legit sources.

The dumber search is, the easier it is for users to ignore the crap and find what they're looking for, even if it's not the first result. The "smart" omission of results has the effect of removing a lot of useful ones too. Google's claims that they have X number of results for a search is essentially meaningless if you can't actually see them all.

Re: The internet is an SEO landfill

#239

Earlier quoted context omitted.

Commerce -> revenues -> viable businesses -> sustainable jobs -> peaceful livelihood prospects. No commerce -> no jobs -> unrest -> violence and wars (citation: centuries of human history). Maybe, commerce is the nail which constrains humanity both in a good and a bad sense - by providing a peaceful alternative to violent lifestyles and by creating a consumerist culture?

When there's a trade that makes both sides better off that increases the happiness in the world. It's a Pareto improvement. When marketing convinces someone to buy something that ends up being a bad trade (i.e. the person that buys it is not made better off) then there is no such improvement and the world is not made better off.

This hints at a common sentiment I see on HN. The idea that everything you and I buy and do is very considered and intelligent. But the great unwashed masses don't put such thought into their purchases, and are being "duped" by evil conspiratorial marketing. If only everybody else was as smart as us, right?

Have you considered that maybe the majority of successful companies in the world are fulfilling customer wants and needs and thus making what you call Pareto improvements? And maybe it's just that the other "dumb people" derive joy from different stuff?

The market sorts these things out. It is mostly efficient. Not perfectly. But mostly. A company that is providing net-negative value to the world is a short-lived company.

Re: The internet is an SEO landfill

#240
post #62
post #18

Search results are so heavily weighted toward commerce, products and services because those are the people that can spend money on SEO. It's made the internet, seen through the eyes of search results, seem like an aggressive, shady market bazaar. I try searching for anything remotely bike related, bike community, mechanical information, or just general cool bike stuff and I can't find the human community underneath a…

Our culture is a frog nailed to the floor. The nail is commerce. The frog can still sorta function. Wiggle in little circles. It ain't pretty or healthy or free. No hopping. I wonder what it would be like to not have that nail through us.

[deleted]
Post reply on HN