Live data from Hacker News

There's a new "Google Jail" for independent wikis

weirdgloop.org

61–70 of 237 posts

Re: There's a new "Google Jail" for independent wikis

#62
post #20

I am slightly on the fence about this. If the goal is simply to provide a good platform for people to create and share information with each other, then it shouldn't matter if the wiki is indexed in Google or not, in fact "invisibility" might even increase the quality of contributions. OTOH if the goal is to monetize other people's contributions then yes, I totally get why the "Google jail" would be bad for that, but…

I think this logic falls apart if you realize that wikis only "work" when you're able to get some of the readers to turn into long-term contributors, and losing 90% of your traffic by not showing up on Google is a great way to kill that funnel. There's a ton of historical evidence of wiki migrations (1) starting out with all of the editors and none of the readers, (2) not doing anything to get those readers to the ne…

Yes, but the point is that maybe in 2026 we should be looking at ways to pay contributors/creators, rather than steal from them (slightly hyperbolic, but you get the gist)

Re: There's a new "Google Jail" for independent wikis

#63
post #5

This is a shame because Fandom is frequently terrible, but also consumes all the oxygen.

It depends on which part. It is certainly heading down the same route as Wikipedia with all the problems that entails. You're usually better off on a smaller Fandom wiki as it won't have beeb taken over yet. The James Bond wiki is particularly bad, but Memory Alpha seems to be better run.

> Memory Alpha seems to be better run.

Some few months ago I couldn't find actors credits for particular roles they had in Star Trek franchise. For whatever reasons these bits were removed but still were present within Wayback Machine.

Then again, in the past I've seen hostile take-overs of fan made wikis into wikia/fandom "infrastructure". In one case wiki was copied, in time articles were slightly edited and new content was added containing... unsolicited fan theories.

The other case: wiki was copied 1:1 due to some inside disagreements, then incorporated into fandom and in approx. 2 years abandoned. The original died as well but someone managed to restore it from Wayback - this time in read-only mode

Re: There's a new "Google Jail" for independent wikis

#64

For a while, Fandom was having annoying ads and content that would get past my ad blocker. But a recent update has caused them to block users with ad blockers. I've been looking for explicit alternatives to Fandom for some content and it's annoyingly sparse. Just the other day I saw some meme about how if we saw as many ads taking over sites like we do now, that was a sure sign of having your machine getting infected…

Given that these ad networks have actually served real malware on normally trustworthy sites (even nytimes was affected) the meme is essentially true even today.

Ad blockers are malware protection.

Re: There's a new "Google Jail" for independent wikis

#65

> Subdomains of existing domains are totally fine Damn, another push to centralizing and walling off user content in 'fandoms', 'reddits' and other closed communities? Have the internet cards been handed out now, and are we in the end-game? Will old domains resell for premiums like low-background-radiation steel?

That or Google is getting so unusable even Discord is a better search engine.

Re: There's a new "Google Jail" for independent wikis

#66
post #33

Earlier quoted context omitted.

That's every company

It’s not every company having the effective monopoly on what can be found (i.e. what exists ) on the web.

Every company is in it for the profits and benefits of its shareholders. Its idiotic to think that companies have benevolent intent toward their customers. If that wasn't the case they would go out of business.

Re: There's a new "Google Jail" for independent wikis

#67

The bar has been raised on all web content for Google to index, esp new domains like you say but even some subdomains. That answer is quality content (higher than AI or human medians), links and pr and social activity and mentions, and then user experience.

And yet, I have to dig my way through pages upon pages of AI trash sites for many Google queries. Somehow THOSE don’t seem to have a problem getting indexed (Maybe they run a sufficient number of ads?).

Given none of the wikis pointed to in the article have ads, I'd say you're right.

Re: There's a new "Google Jail" for independent wikis

#68
post #43

Earlier quoted context omitted.

> But a recent update has caused them to block users with ad blockers. I'm noticing that Ad-Shield is becoming a continual fight these days with pageloads that complete then they fly in with a "ha ha here's a fake error report turn off your adblocker aye?". Every time their live support chat is open (why would the anti-adblock firm even offer one), I send them Goatse.

> Every time their live support chat is open […], I send them Goatse. Unfortunately there is little chance that a human is anywhere near the other end of that chat these days, so your evil is somewhat wasted. > (why would the anti-adblock firm even offer one) They want to appear to care. And appear to be a company of real people, who you might like if you met in real life, who will be out of pocket if you don't look…

If there is a person on the other end it’s not someone who has any link to the company, they’ll be an outsourced service, so all you’d be doing is sending graphic porn to a poorly paid worker.

It’s like screaming at a cashier because the supermarket head office made annoying changes to the company.

Re: There's a new "Google Jail" for independent wikis

#69
Some of this might just be problems with the websites themselves. With the Path Of Exile 2 Wiki example, I'm looking at the sitemaps. Once you get through the first sitemap of nested sitemaps (ugh, not a fan of those), the very first actual sitemap returns:

  XML Parsing Error: no root element found
  Location: https://www.poe2wiki.net/sitemap/sitemap-poe_wiki-poe2_wiki--NS_0-0.xml
  Line Number 18040, Column 1
(EDIT: Of course, after I post this, it's now working again...)

On to the Hytale Wiki example. I don't know if what I'm about to say applies to Google, but I'm approaching it from the perspective of my tiny dumb indieweb indexer for my personal search engine. It is much easier for me to index from a sitemap rather than try to crawl a website, so I basically look exclusively at sitemaps.

Looking at Hytale Wiki, my process:

* Site has a robots.txt file - good!

* Robots.txt mentions a sitemap - excellent!

* The sitemap is stored at /images/sitemaps/index.xml ... oh. I would usually not index anything from a /images/ folder, because I want to index pages only, not images. This would likely trip my exclusion filters. Let's ignore that and continue.

* The sitemap is a list of an additional 26 nested Gzipped sitemaps. Oh.

This is the point where my tiny dumb indexer would stop. Gzipped sitemaps are part of the standard, but they're relatively rare on the web for small sites. They typically only get used if a sitemap file exceeds the 50,000 pages-per-sitemap limit. In this case, 26 * 50,000 makes the HyTale Wiki look like a 1.3 Million page site. Do I really want to index 1.3 Million pages, an estimated 13GB of indexed text data, about a video game I'll probably never play?

My search index is storage constrained, and my indexer is very time constrained. The time I spend indexing your site is time not spent indexing another, possibly higher quality website. So at this point, I'd just grab the front page and disappear... like Google apparently does too.

Of course, the HyTale site isn't 1.3 Million pages, it's only 4,277 articles. That would all fit in the root sitemap file, and that might be the better approach for getting indexed.

Re: There's a new "Google Jail" for independent wikis

#70

> Subdomains of existing domains are totally fine Damn, another push to centralizing and walling off user content in 'fandoms', 'reddits' and other closed communities? Have the internet cards been handed out now, and are we in the end-game? Will old domains resell for premiums like low-background-radiation steel?

> Will old domains resell for premiums like low-background-radiation steel?

This has been a thing for a long time in domain name resale for speculation or link farming or similar, domains that are aged with "backlinks" sell for more money.

https://www.google.com/search?&q=domain+resale+backlinks

Post reply on HN