Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's a new "Google Jail" for independent wikis

weirdgloop.org

Post reply on HN
Print view View original

191–200 of 238 posts

Re: There's a new "Google Jail" for independent wikis

#191

Post by arlattimore »

SyneRyder wrote: post #69

Some of this might just be problems with the websites themselves. With the Path Of Exile 2 Wiki example, I'm looking at the sitemaps. Once you get through the first sitemap of nested sitemaps (ugh, not a fan of those), the very first actual sitemap returns: XML Parsing Error: no root element found Location: https://www.poe2wiki.net/sitemap/sitemap-poe_wiki-poe2_wiki--NS_0-0.xml Line Number 18040, Column 1 (EDIT: Of c…

The sitemap being hosted at /images/sitemaps/index.xml is a problem. Technically, that sitemap can only include URLs beginning with /images/sitemaps/. They should move that file to the root, like /sitemap_index.xml and reference /sitemap-1.xml, /sitemap-2.xml etc.

Re: There's a new "Google Jail" for independent wikis

#192

Post by nostrademons »

JayNeely wrote: post #117

The article says this isn't an issue for subdomains of established domains, but the issues it describes match what we experienced setting up a game wiki for Cyber Knights: Flashpoint at https://cyberknightswiki.tresebrothers.com/ -- almost a year of not having any pages show up in search results, even though we were getting good links to it and there were no better alternatives that should have ranked (we never start…

It used to be (late 00's) that wikis had a big advantage in search results because backlinks were weighted heavily by the ranking algorithm and the dense connectivity of wikis meant that there were always a zillion backlinks to pages on a wiki.

Entirely possible that this is caught in a malfunctioning webspam filter. A wiki is largely indistinguishable in structure from an SEO link farm; indeed, any abandoned wiki can be and usually is turned into a link farm by bots. Maybe Google decided that its default was going to be to ban all wikis until a sufficient mass of evidence comes that it's trusted. Which, again, sucks for search quality, but the web and Google Search have both been pretty much dead for the last ~5 years anyway, we just rely on them anyway.

Re: There's a new "Google Jail" for independent wikis

#193

Post by jimnotgym »

swed420 wrote: post #186
jimnotgym wrote: post #179

Earlier quoted context omitted.

My incentive was merely to catalogue some historical things that were not catalogued elsewhere. Google, Wikipedia etc. LLMs knew only what was on Wikipedia. I don't mind that LLMs will find it....I used a bunch of agents to find sources for them and produce summaries of it, and even to build the site. It took a lot of human research too, but all my notes and snippets were sent to LLMs. I hope this means no more of th…

My goal was to share complex things I've learned through IRL experience, and explain them in detail. Unfortunately in my case, I can't justify the distilling/production time sink unless there is some kind of monetary incentive. I already donate time to other projects, and there's only so much time in a day. YMMV

Absolutely, and it wasn't a criticism. Just to point out there are other goals available!

Re: There's a new "Google Jail" for independent wikis

#194

Post by SyneRyder »

jimnotgym wrote: post #170
SyneRyder wrote: post #56

Earlier quoted context omitted.

Have you added a robots.txt and a sitemap.xml, even an RSS file? While I don't know if this will help with Google, I have my own tiny dumb search indexer, and a sitemap is by far the easiest way for me to index an entire website. I'll discover the sitemap from reading your robots.txt file. RSS helps too, but my indexer uses that mostly to find fresh pages without going through your entire sitemap again. I know Kagi's…

Sitemap was submitted and search console says it is good on day one. Robots.txt is in place I think of RSS as more of a bloggers tool...am I thinking of this wrong.

If you're only concerned about Google, then that's probably all you need to / can do. It won't even matter to Google whether or not you put a "Sitemap:" line in your robots.txt if you submitted it directly with them.

As for RSS, Google do regard RSS as another sitemap format. There's info here:

https://developers.google.com/search/docs/crawling-indexing/...

I would not submit an RSS file to Google as your sitemap, stick to XML. But also having RSS can be useful for search indexers, because in one RSS download they can find which pages are new, and can index the actual relevant content of the page without all the ads & other cruft.

An RSS feed is really like a newsletter about your website. So if there's anything where visitors to your website might find value in keeping updated via an email newsletter, consider putting that same "newsletter" content also online, and making that your RSS feed. If the content in your RSS feed contains links to the other pages of your website, that can help with page discovery etc. Something like "we just updated the XYZ product page, check it out!" is zero value, but if you've got an article about "Here's how the brand new ABC feature we just added XYZ Product works", then linking to XYZ Product from within the article & RSS content will make sense. It helps the search engines, but it also helps the user who might want to click through to the product that your article is about.

Re: There's a new "Google Jail" for independent wikis

#195

Post by mitxela »

port11 wrote: post #161
raincole wrote: post #100

When I still payed Dota, there was a wiki hosted on fandom. The experience was terrible. It wasn't always update-to-date with the latest patch, and on mobile there were so many ads rendering the whole thing unreadable. The catch is that there was a better wiki, called Liquipedia. But Google never showed it first and I kept getting tricked to click that shitty fandom site. (It was years ago. But I just tested it a bit…

Fandom is an abomination and filled with irrelevant or very basic content for most games. It wouldn’t exist if not for its phenomenal ranking on game-related searches…

I think their trick was to buy something that was legitimately highly rated, then enshittify it.

Re: There's a new "Google Jail" for independent wikis

#196

Post by mitxela »

whstl wrote: post #183
raincole wrote: post #100

When I still payed Dota, there was a wiki hosted on fandom. The experience was terrible. It wasn't always update-to-date with the latest patch, and on mobile there were so many ads rendering the whole thing unreadable. The catch is that there was a better wiki, called Liquipedia. But Google never showed it first and I kept getting tricked to click that shitty fandom site. (It was years ago. But I just tested it a bit…

When Jimmy Wales launched Fandom, then called Wikia, a shit-ton of (actually encyclopaedic) content about fictional stuff was deleted from Wikipedia to be moved to Fandom, with the excuse that it was a "sister project", that it was the right place for things to be moved, etc. Fandom was pure corruption even before the tenth round of enshittification.

Are you trying to say Jimmy Wales is corrupt because he didn't let you have a separate page for every Lord of the Rings character?

Re: There's a new "Google Jail" for independent wikis

#197

Post by mitxela »

charcircuit wrote: post #169
raincole wrote: post #100

When I still payed Dota, there was a wiki hosted on fandom. The experience was terrible. It wasn't always update-to-date with the latest patch, and on mobile there were so many ads rendering the whole thing unreadable. The catch is that there was a better wiki, called Liquipedia. But Google never showed it first and I kept getting tricked to click that shitty fandom site. (It was years ago. But I just tested it a bit…

>It wasn't always update-to-date with the latest patch In case you didn't know Fandom is not the one who is writing the wiki. The pages are community contributed. If it's out of date it means no one has contributed information about newer patches.

This doesn't actually matter, the user experience is just as bad. If you want visitors then you have to find a way to fix it.

Re: There's a new "Google Jail" for independent wikis

#198

Post by panarky »

mitxela wrote: post #133
JayNeely wrote: post #117

The article says this isn't an issue for subdomains of established domains, but the issues it describes match what we experienced setting up a game wiki for Cyber Knights: Flashpoint at https://cyberknightswiki.tresebrothers.com/ -- almost a year of not having any pages show up in search results, even though we were getting good links to it and there were no better alternatives that should have ranked (we never start…

The problem is that independent wikis don't pay the fandom tax to google for preferred search placement. Have you tried contacting your local corruption bureau to negotiate a rate tailored for your circumstances?

Payment for search placement is not a thing.

Re: There's a new "Google Jail" for independent wikis

#199

Post by mitxela »

nicce wrote: post #49
AlAjem wrote: post #45

Earlier quoted context omitted.

>around decade ago Two decades now actually. By 2016 we were already seeing the mass transition to browsing on phones instead of PCs, Windows 10 was already out and for reasons I am not quite sure of the old homepage and toolbar hijackers were on the decline and adblockers were on the rise although I am sure there were still people using malware ridden Internet Explorers for years after. https://en.wikipedia.org/wiki…

Ouch, already 11 years since the Windows 10 release.

It really was the last Windows anyone with a brain would ever care about.

Re: There's a new "Google Jail" for independent wikis

#200

Post by Melatonic »

lqstuart wrote: post #177
no_wizard wrote: post #126

Earlier quoted context omitted.

Shoutout to Kagi search for having a deny list feature. It’s really nice for this sort of thing.

I honestly love Kagi, it's like every single "it would be nice if Google gave me feature X instead of being greedy" in one place

Exactly. And SmallWeb is awesome too
Post reply on HN