The Return of the 90s Web

How about TF(PageRank)/IDF(PageRank)? As in, start by ranking the individual result URLs with their PageRank for the given query; but then normalize those rankings by the PageRank each TF(PageRank) result URL’s domain/origin has for all known queries (this is the IDF part.)

Then, the more distinct queries a given website ranks for (i.e. the more SEO battles it wins/the more generally optimal it is at “playing the game”), the less prominently any individual results from said website would be ranked for any given query.

So big sites that people link to for thousands of different reasons (Wikipedia, say) wouldn’t disappear from the results entirely; but they would rank below some person’s hand-written HTML website they made 100% just to answer your question, which only gets linked to on click-paths originating on sites that contain your exact search terms.

This would incentivize creating pages that are actually about one particular thing; while actively punishing not just SEO lead-gen bullshit; not just keyword-stuffed landing pages we see in most modern corporate sites; but also content centralization in general (i.e. content platforms like Reddit, Github, Wikipedia, etc.) while leaving unaffected actual hosting by these platforms, of the kind that puts individual sites on their own domains (e.g. Github Pages, WordPress.com, Tumblr, specialty Wikis, etc.)

———

A fun way to think of this is that it’s similar to using a ladder ranking system (usually used for competitive games) to solve the stable-marriage problem on a dating site.

In such a system, you have two considerations:

• you want people to find someone who’s highly compatible with them, i.e. someone who ranks for their query

• you want to optimize for relationship length; and therefore, you want to lower the ranking of matches that, while theoretically compatible, would result in high relationship stress/tension.

Satisfying just the first constraint is pretty simple (and gets you a regular dating site.) To satisfy the second constraint, though, you need some way of computing relationship stress.

One large (and more importantly, “amenable to analysis”) source of relationship stress, comes from matches between highly-sought-after and not-highly-sought-after people, i.e. matches where one partner is “out of the league of” the other partner.

So, going with just that source for now (as fixing just that source of stress would go a long way to making a better dating site), to compute it, you would need some way to 1. globally rank users, and then 2. measure the “distance” between two users in this ranking.

The naive way of globally ranking users is with arbitrary heuristics. (OKCupid actually does this in a weak sense, sharding its users between two buckets/leagues: “very attractive” and “everyone else.”)

But the optimal way of globally ranking users, specifically in the context of a matching problem, is (AFAICT) with IDF(PageRank): a user’s “global rank” can just be the percentage of compatibility-queries that highly rank the given user. This is, strictly speaking, a measure of the user’s “optionality” in the dating pool: the number of potential suitors looking at them, that they can therefore choose between.

If you put the user on a global ladder by this “optionality” ranking; and normalize the returned compatibility-query result ranking by the resulting users’ rankings on this global “optionality” ladder; then you’re basically returning a result set (partially) optimized for stability-of-relationship: compatibility over delta-optionality.

———

All this leads back to a clean metaphor: highly-SEOed websites—or just large knots of Internet centralization—are like famous attractive people. “Everyone” wants to get with them; but that means that they’re much less likely to meet your individual needs, if you were to end up interacting with them. Ideally, you want a page that’s “just for you.” A page with low optionality, that can’t help but serve your particular needs.

So, basically Yacy?

https://yacy.net/

Sure, but you can just as easily fall into the trap of massively over-engineering your client-side rendering because you just grab packages off the shelf for every little thing instead of going "it's a little bit more work, but there is no reason to any of this when the browser already does this", like ending up with a 100kb bundle for what is effectively just a standard form that would work better, with less code, if it just used a normal form with built-in validation for almost everything, with the final acceptance/rejection determined by the api endpoint.

It really depends on which SSR approach we're comparing to which client-rendering approach, and who you're optimizing for.

> It's not shocking that your modern iPad renders pages faster than the model released a decade prior. Not only does it have far more power and memory but the network (both last mile and far end) is faster. It's also got an extra decade of development on WebKit. The web is more bloated but the modern iPad has ramped up its power to compensate.

The main point was that the 2010 iPad was being given the most favorable conditions, and it still lost horribly, because even compared to contemporaneous devices, it was very underpowered, unlike current iPads:

- your claim is that websites are substantially heavier now (which I agree), putting the 2018 iPad at a disadvantage

- the 2010 iPad was browsing early 2010 websites in that video, so we've had 10 years of bloatification since then

- the 2018 iPad Pro is browsing 2020 websites, websites built years after it was released, so surely more "bloated" than they were in 2018

- being 2010 websites, they were probably much simpler to render

- the 2010 iPad's screen had 5x fewer pixels to contend with

Nowhere was I saying that the 2010 iPad was super slow to render 2020 webpages in all their bloaty goodness. That would be an obvious conclusion. If the 2010 iPad's performance was so good at the time, but only became slower as the web became much more bloated, why was it still so much slower at browsing 2010 websites than my 2018 iPad is at browsing 2020 websites?

The 2010 iPad was actually slow from the beginning, as the video proves. Since it was slow back then, it shouldn't be surprising that it's slower and more painful now that websites want to support higher resolution experiences by default. Yes, they could put effort into giving old devices a lower res experience, but why? That old browser is one giant security vulnerability at this point, and no one should be browsing any websites they don't control on that thing.

Even with all those advantages being in the 2010 iPad's court, it was still 3x to 5x slower than a 2018 iPad browsing 2020 websites at 5x the resolution. This is not even a 2020 iPad Pro -- this is a 2018 iPad Pro. Imagine how much worse a 2008 iPad would have been at browsing 2010 websites, if it had existed.

You say that it's "not shocking" that they ramped up the power so it can browse better, but the point is that we're loading substantially heavier websites today significantly faster.

How is that possible? Because the 2010 iPad was severely underpowered. If it had been running on a chip that was equivalent to laptop processors of the era (as my iPad Pro's chip is), then it would likely have loaded the 2010 websites about as quickly as my iPad is loading 2020 websites.

> Once a page was loaded and rendered the scrolling, tapping links, and interacting with forms was all usable fast. Even while the content was loading you could interact with the page.

Yes, it's very impressive how much interactivity Apple was able to give the 2010 iPad with its really terrible processor, once the loading finished.

That interactivity isn't because the chip was any good. I remember very clearly that it was because you were basically scaling a 1024x768 PNG while you zoomed in and out. Once you let go, the iPad would take a second to re-render the page at the new zoom level, but you were stuck staring at a blurry image for a second after zooming in. The GPU was really good at scaling a small image up and down. The CPU was not so good at rendering websites.

It was also very easy at the time to scroll past the end of the pre-rendered image buffer, and you would just stare at a checkerboard while you waited on the iPad to catch up and render the missing content. iOS actually drastically limited the scrolling speed in Safari for many years to make it harder for you to get to the checkerboard, but it was still easy enough.

> Look at Reddit versus old.reddit.com. The "modern" Reddit page has poor interactivity even on my current iPad. The old.reddit.com site, which is similar in complexity to 2010's Reddit, renders damn near instantly and has no interactivity issues.

New Reddit is one of the worst websites on the entire internet right now, if not actually the worst popular website in existence. I really don't understand how that hasn't been scrapped at this point. It's not representative of modern web experiences, except possibly in your mind. YouTube, The NY Times, Facebook, Amazon... these are all modern web experiences that work great on anything approaching reasonable hardware.

Personally I'd stick with the static site generators.

If you get something like wordpress installed on your own server, you now need to then commit doing the upgrades for it. I've been burnt enough times with self-hosted things like this where I missed an upgrade for whatever reason (real-life etc) and the site was hacked very, very quickly. There are ones that auto-update themselves but I have found that those have their own issues so cannot be relied upon.

Static site generators tend not to have this problem.