Some stream of consciousness thoughts on the history of internet communities, particularly those centred around tech.
Usenet had immense value in well defined subgroups prior to the Eternal September (and for some time after, regardless of what people may say). IRC ha(s|d) similar values, and remains a force within niche communities on the tech side. Slashdot was an early mover in the moderated community space which had to arise from the newfound populist web.
I still think /.'s comment moderation was superior to the HN system (pre and post-visible comment scores), but the firehose was too late and too poorly implemented to solve editorial issues.
In the middle of this, Kuro5hin rose and fell, metafilter grabbed some component of the serious moderated discussion which it still retains. Fark came and went. Boingboing, SA, b3ta. All significant for a time but not names on people's lips today.
HN cannibalised a significant portion of /., but failed to convert the greybeards - the discussion here is noticeably different because of it (and lacking the perspective sometimes).
Digg suffered greatly from demagogues (as does HN to an extent), descended too rapidly into linkbait and celebpop trash, and fell to Reddit. The redesign was just the nail in the coffin of an already dead community.
Reddit became a very granular experience from its initial tech focus, with a current frontpage of dubious intellectual interest, but their popularity speaks wonders for the ad-hoc community created by diverse interest groups with a common central park. They struggle with discovery for new members, and an apparently descending base age group.
Communities come and go. Small herds migrate towards the latest point of interest and some stick. Groupthink is a large driver of community malaise, certainly within the tech discussion arena. Individuals dominate submissions and discussions and evolve to minor demagogue status. Some communities evolve to tackle a smaller arena than just the general topical discussion field, but topicality remains critical.
Quora has tackled 'big answers'. StackOverflow 'correct answers'. These are some minor elements of the value of the larger communities, much in the same way that Hipmunk, AirBnb etc have abstracted value away from Craigslist. Hyperlocal is the next big thing with FrontPorchForum and NextDoor tacking non-technical local discussion.
I still view the approaches to these problems as relatively unsolved and ripe for disruption, in particular the algorithms related to subject and comment popularity, user 'karma' (for better or worse), and approachable comment threading when a userbase grows beyond the 'scan a single page' scale. I'm not convinced that a one-size-fits-all approach will ever work, but even within niche tribes there remains a problem with staying 'current' while avoiding alienating the 1-2% who drive much of the discussion.
I fully expect a new dominant discussion forum to arise in the tech scene in the next couple of years, but Lobsters seems to be a kneejerk reimplementation of HN that even if it claws some traction would have to evolve rapidly to solve problems rather than dangling the 2013 model of a 2012 carrot.