Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does Pinterest dominate Google text search results?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 329 posts

Re: Ask HN: Why does Pinterest dominate Google text search results?

#41

Post by djrobstep »

burnthrow wrote: post #29
knuckleheads wrote: post #25

Earlier quoted context omitted.

We are pursuing ideas that we think will increase competition in the search engine market. Our main focus right now is making Google’s index of the web available for use by competitors, since no one other than Google is really allowed to crawl the web. If people were allowed to use the index to build what they want, then you or somebody else would be able to build a search engine that avoids the Pinterest cruft and w…

> making Google’s index of the web available for use by competitors What incentive would Google have to continue populating that index? Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it? > since no one other than Google is really allowed to crawl the web Maybe this is the problem that needs solving.

> What incentive would Google have to continue populating that index?

Not going to jail, presumably.

Re: Ask HN: Why does Pinterest dominate Google text search results?

#42

Post by knuckleheads »

aleph_naught wrote: post #30
knuckleheads wrote: post #25

Earlier quoted context omitted.

We are pursuing ideas that we think will increase competition in the search engine market. Our main focus right now is making Google’s index of the web available for use by competitors, since no one other than Google is really allowed to crawl the web. If people were allowed to use the index to build what they want, then you or somebody else would be able to build a search engine that avoids the Pinterest cruft and w…

> since no one other than Google is really allowed to crawl the web. ??

There are two main reasons why I say nobody besides Google is really allowed to crawl the web.

The first is that Google gets much more access to pages on websites than everybody else. You can see this by examining the robots.txt files of various websites[0]. I've been doing this for several years now and Google has a consistent advantage across many thousands websites that I've looked at. This adds up to a significant advatnage and many search engine operators complain about how it hampers their ability to compete with Google[1].

The second is that Google gets to ignore crawl delay directive in robots.txt while other search engines don't[2]. Website operators cannot tell Google how fast they want their website crawled, they can only request that Google slow down. If another search engine tried to do what Google does, they would likely be blocked by many important websites.

If you would like to read more about this, please checkout https://knuckleheads.club/

[0] https://pdf.sciencedirectassets.com/robots.txt

[1] https://www.nytimes.com/2020/12/14/technology/how-google-dom...

[2] https://www.seroundtable.com/google-noindex-in-robots-txt-de...

Re: Ask HN: Why does Pinterest dominate Google text search results?

#43

Post by [deleted] »

aleph_naught wrote: post #30
knuckleheads wrote: post #25

Earlier quoted context omitted.

We are pursuing ideas that we think will increase competition in the search engine market. Our main focus right now is making Google’s index of the web available for use by competitors, since no one other than Google is really allowed to crawl the web. If people were allowed to use the index to build what they want, then you or somebody else would be able to build a search engine that avoids the Pinterest cruft and w…

> since no one other than Google is really allowed to crawl the web. ??

[deleted]

Re: Ask HN: Why does Pinterest dominate Google text search results?

#44

Post by Winterflow3r »

knuckleheads wrote: post #12

One big reason why Pinterest ranks so highly, and why you are seeing different tld's in the results, is because they have a variety of tld's that they use and those tld's all have separate crawl budgets from each other. So Google crawls Pinterest about an order of magnitude more than most other sites. You can read more about this here: https://medium.com/pinterest-engineering/how-switching-our-d... Source: I run a th…

This I did not know about and I'm surprised they don't get penalised by Google in some way at the display of search results stage. I mean sure, I suppose one can do the -pinterest trick, but it's still a bit weird that Google thinks search results that are pretty much the same site but from different tlds are not spammy. Do you think they will change this to prevent Pinterest from essentially spamming some search queries by using the tld hack?

Re: Ask HN: Why does Pinterest dominate Google text search results?

#45

Post by amelius »

derenge wrote: post #2

“Deconstructing Pinterest’s reverse-image-search SEO growth hack” https://www.rankscience.com/blog/pinterest-image-seo-growth-...

Ugh, this is bad. Makes you wonder, why they let this happen. Do people at Google own Pinterest stocks?

Re: Ask HN: Why does Pinterest dominate Google text search results?

#46

Post by hftf »

Some relevant prior discussions:

https://news.ycombinator.com/item?id=21622322 (Nov 2019) "Tell HN: Google should drop Quora from search results" 1000+ upvotes

https://news.ycombinator.com/item?id=16613996 (Mar 2018) “Pinterest needs to be removed from Google IMO” 1100+ upvotes

https://news.ycombinator.com/item?id=16388833 (Feb 2018)

And many more: https://www.google.com/search?q=site%3Anews.ycombinator.com+...

Re: Ask HN: Why does Pinterest dominate Google text search results?

#47

Post by maya24 »

blunte wrote: post #13
hayksaakian wrote: post #3

There's no intentional or manual effort at Google to promote pinterest. Pinterest shows up because they understand how the Google algorithm works and built their website to display all the signals that Google looks for in relevant image content. They understand user intent and generate URLs that present content in a way that google expects to see. Examples of how they do this from their engineering team: https://medi…

Google should be asking from a product perspective if they are providing the search results the user is most likely seeking. I seriously doubt that most pinterest results are what users are seeking. As such, Google probably should consider changes to their algorithm to compensate (cat and mouse) for pinterest's gaming. That is assuming how pinterest does it is not against any google rules... because if it were agains…

Stop using Google search. DDG works just as well

Re: Ask HN: Why does Pinterest dominate Google text search results?

#49

Post by Winterflow3r »

derenge wrote: post #2

“Deconstructing Pinterest’s reverse-image-search SEO growth hack” https://www.rankscience.com/blog/pinterest-image-seo-growth-...

Oh wow :( I'm a bit scared by this, because of what it means for smaller competitors, who can't afford to pour investor dollars into hiring a whole team of software engineers just focused on SEO driven growth

Re: Ask HN: Why does Pinterest dominate Google text search results?

#50

Post by [deleted] »

knuckleheads wrote: post #34
burnthrow wrote: post #29

Earlier quoted context omitted.

> making Google’s index of the web available for use by competitors What incentive would Google have to continue populating that index? Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it? > since no one other than Google is really allowed to crawl the web Maybe this is the problem that needs solving.

> What incentive would Google have to continue populating that index? Presumably they would still want to run google.com and make money off of it. > Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it? No. You would not get the advantage that Google gets when it crawls the web and so would not have access to a large amount of data that nobody else has access to.…

[deleted]
Post reply on HN