Stories that Hacker News removes from the front page

I don't really read this terribly as complaining. It is an interesting analysis of some posts (including his own) that seemed to behave atypically in a non-transparent manner to the user base, that may suggest some manipulation of the ranking. At no point does it seem to take any accusatory tone, or complain. The hypothesis may be incorrect, but it is not unreasonable, and it does seem relevant to readers of hacker news.

I don't think being "bored" by something is generally considered a good reason to flag content. I don't see how it could possibly be considered off-topic either.

This blog suggest that moderator action must have been necessary, but seems to say that flags are unimportant.

I disagree. Just a few flags can cause a story to drop off the front page.

I can comment on the "Books that Aaron Swartz read, loved and hated" story. OP posted that link to HN and it made the front page. The story was soon flagged by the community (people viewed it as Amazon affiliate spam) and it fell off the rankings. When the moderator dang noticed that the story had been flagged, he decided to restore it (https://news.ycombinator.com/item?id=13840869). So yeah, that explains why this particular story fell off the front page quickly but does not have any [flagged] or [dupe] tags attached now.

I've learned that it's best not to jump to conclusions based on what you think is true (however sound your analysis might be). Always ask the other side(s) for an explanation. In this case, you could have sent an email to the mods asking for an explanation. If you find their response unsatisfactory, go ahead and write a post explaining why.

I've got my account (with 9+k point) banned. Which is, of course, annoying, but the most annoying thing was the comment left by moderator "we banned". How about "I banned"? Faceless corporate "we" does not look well. There was also a lie about warning which never happened, but hey, "not lying" is not in the guidelines, so… And while some of my comments might be harsh I sometimes wonder if overzealous moderation leads us to creation of some kind of Stepford here.

I agree with you overall regarding transparency. It would be nice to see removed pages listed somewhere, or perhaps just the fact that it was done with a counter and maybe a timestamp. But as someone who reverse engineers a new mobile app nearly once a week these days: candidly, you can summarize nearly the entire discipline as security through obscurity :).

You use the word "suppress", but we generally can't use words stronger than that in security. We deal in abstractions that are hard to reasonably quantify, so we do so by approximations of monetary or temporal costs, and frequently both. We also try to limit absolutes. Reverse engineering exists as a discipline because 1) contrary to popular opinion, security through obscurity is valid, if incomplete and 2) there's frankly not much more that you can do in many cases other than obfuscation.

There are situations where algorithm secrecy gains you nothing defensively and is actually a strategic disadvantage, such as in encryption or hashing. But in situations where you fundamentally cannot discriminate between authorized users, such as in email (spam), search results (SEO) or, here, Hacker News (front page), you cannot rely on the strength of the algorithm to properly discriminate between users, because that's not its intended purpose. In these situations, obscurity is essentially your only remaining option.

To be fair the Hacker News moderators have more control of the ranking algorithm being reversed as it's on a remote server they control, as opposed to embedded in a client deployed to inherently untrustworthy hands. And once the information is out, it's out. But I don't agree that they have a trust or transparency imperative to keep that sort of submission on the front page. Even if the information exists, there's no reason to make it even more accessible. They can remove it and also improve the ranking algorithm.

If you want to design a general purpose web application without significantly reducing usability, functionality that is not restricted through authentication or higher levels of authorization is susceptible to reverse engineering. Being that the ranking algorithm is not client-side, there are fundamental protections we cannot bypass, but much of it is still inherently obfuscation. There is rate-limiting of course, and you have to log in, but the inherent inputs and outputs can still be somewhat flexibly assessed over reasonable timespans because there are hard usability requirements in place.

The tl;dr: algorithms which cannot be gamed because their inputs have significant quantifiable and controllable time/monetary costs do not require secrecy - these are excellent for implementing authorization. Algorithms which do not have such costs are not appropriate for authorization and, unless also paired with significant authorization constraints, require some degree of obfuscurity.