As you suggest, journals are a critical area for society.
Since we're on HN, we get to take off our technical hat and put on our product hat -- not the fit-to-customer product, but disrupt-the-industry product.
What are the incentive structures that would destroy this or make it work?
Consider the scenarios today...
- What's wrong with the thousands of similar-sounding journals popping up on China publishing research that is either unvetted or copied from other journals, so researchers can satisfy their publish-or-perish needs?
- How do you deal with reviewer networks?
- How do you get the best of the best, people with no time and plenty of opportunities and money, to provide effective feedback on an avalanche of articles?
- How do you get the support of, without being dominated by, the big research companies and universities?
- Look at the history of reputation systems, from quora to stackexchange to games, on a matrix with the difficulty of determining correctness of questions:
-- Who won the game?
-- Does this code work?
-- Which library should I use?
-- Is this paper correct and valuable?
- What about prediction markets, which try to use money/investment as a measure of seriousness: how would that help or hurt? (and isn't that what we're doing by insisting academics publish for tenure?)
In the spirit of MVP, you might consider a pivot to a much smaller problem: how to professional groups establish and document their standard of care? A team wiki? (gets stale, disconnected from operations) Run books? (too stepwise to convey meaning needed to transform the system) Slack-capture? email? documentation?
What about capturing the benefit in a change of format, from opaque text/pdf to something like a per-domain semantic web, with connections to empirical methods and findings? That could provide incentives for everyone. Like the professional-group scenario, the killer feature would be something that grows incrementally with multiple authors (as code does for developers).