Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case for a Git-Powered Project Gutenberg

neosmart.net

Post reply on HN
Print view View original

21–30 of 47 posts

Re: The Case for a Git-Powered Project Gutenberg

#21

Post by jberryman »

I think this is a great idea in general. As an aside, when I first discovered Project Gutenberg I thought it was really exciting, but as I've grown up my perspective has changed: it feels like a desperate attempt to scrape together something like what used to be recognizable as culture from cast-off bits of art that capitalism can no longer commodify.

Same with FLOSS and Creative Commons. They are at the same time wonderful, and yet seem to be the products of a reality in which every relevant part of our so-called cultural identity is owned by a corporation.

I have an idea that the "creative commons" used to exist and that sometime in the 20th century it became hopelessly barren.

Re: The Case for a Git-Powered Project Gutenberg

#22

Post by aiscott »

pronoiac wrote: post #17

I'm not sure git's the best choice for handling multiple gigabytes of data. That estimate's based on their DVD releases. Oddly, I'm not positive that they're compressed. I've seen a proposal for using Usenet (or something like it) for Wikipedia; perhaps that would be appropriate here? Surfacing changes would be easier. And if you didn't want to follow each and every format available, you could. (Having every format i…

I wouldn't think that the whole of PG would be a single repository. What came to mind when I read the article was that each book would be in its own repository, and that seems to me to be well within Git's capabilities.

Re: The Case for a Git-Powered Project Gutenberg

#23

Post by LoonyPandora »

Shove it on GitHub, it's edit in-browser functionality means it is accessible to non-techies. Problem solved.

All it needs is someone to pick a master format for the text and run with it. That should be someone with enough domain knowledge to choose wisely. I would say something like markdown with embedded metadata (like how you format a blog post with jekyll), but I admit I don't know anything about the requirements of the project.

Re: The Case for a Git-Powered Project Gutenberg

#24

Post by urschrei »

ComputerGuru wrote: post #3
zdw wrote: post #2

Version control is superior than other "sync" solutions when the data is: - Modified mainly by humans, at human speeds - Mistakes can be made and need reverted atomically - Data integrity and chains of changes need to be preserved I agree with the article that PG is a great example of where version control could work with great effect. The downside is that teaching non-techies how to make it work is difficult, and th…

I'm the author of TFA and I must completely agree with you on this point. Is there a human-friendly VCS out there? I don't care how featured or not it is, as Git can be configured to pull from it (a la git-svn). Just anything that non-tech-related persons (or even just non-coders!) can use without wanting to pull their hair out and feeling very stupid? For example, one of my software products has translations powered…

Something like Github's "forking with the edit button"[1] functionality would be perfect for this. If you want to make a correction, press a button to see the content you were looking at in a web-based editor. Make your changes, and add your email address to a notification field. Changes go into an approval queue, and you get an email if/when your change goes live. No need to expose the casual user to the perils of DVCS, though of course a "proper" git-based infrastructure would exist in parallel for people who want more control.

[1] https://github.com/blog/844-forking-with-the-edit-button

Re: The Case for a Git-Powered Project Gutenberg

#25

Post by dmlorenzetti »

We’ll be contacting Project Gutenberg with a link to this article...

The article makes good points, but makes them in a way that's likely persuasive only to people who already know about version control. Suppose you represent Project Gutenberg and you've never heard of distributed version control. Here's what you get told:

It’s downright foolish not to take advantage of the wonders a good VS can work with this sort of content: versions are revisions, editions are branches, commit logs preserve integrity and posterity, and an index of all changes is forever kept. Nothing is ever lost or overwritten, and the changes over time can be analyzed, indexed, and reviewed... [N]othing could be easier than forking the original, making your changes, then opening a ticket to propose that PG merge your changes back into their "official" distribution!

Some of this will sound interesting, but most will sound like gobbledygook. I also doubt whether the paragraph that follows-- which describes how a DVCS will make it easier for people who intentionally corrupt works found on PG to benefit from the future work of PG members-- will excite the sorts of people who rise to the level of decision-maker at Project Gutenberg.

In short, this is good advocacy for getting people who know about DVCS to help out PG, but it probably isn't good advocacy to get PG interested in DVCS.

Re: The Case for a Git-Powered Project Gutenberg

#26

Post by ComputerGuru »

dmlorenzetti wrote: post #25

We’ll be contacting Project Gutenberg with a link to this article... The article makes good points, but makes them in a way that's likely persuasive only to people who already know about version control. Suppose you represent Project Gutenberg and you've never heard of distributed version control. Here's what you get told: It’s downright foolish not to take advantage of the wonders a good VS can work with this sort o…

The email I sent included several paragraphs description of the benefits to PG and the problems that DVCS would solve.

Re: The Case for a Git-Powered Project Gutenberg

#27

Post by pnathan »

ComputerGuru wrote: post #3
zdw wrote: post #2

Version control is superior than other "sync" solutions when the data is: - Modified mainly by humans, at human speeds - Mistakes can be made and need reverted atomically - Data integrity and chains of changes need to be preserved I agree with the article that PG is a great example of where version control could work with great effect. The downside is that teaching non-techies how to make it work is difficult, and th…

I'm the author of TFA and I must completely agree with you on this point. Is there a human-friendly VCS out there? I don't care how featured or not it is, as Git can be configured to pull from it (a la git-svn). Just anything that non-tech-related persons (or even just non-coders!) can use without wanting to pull their hair out and feeling very stupid? For example, one of my software products has translations powered…

I would recommend Mercurial for such a project. It's much more graspable by non-nerds than git IMO.

Re: The Case for a Git-Powered Project Gutenberg

#28

Post by lazerwalker »

ComputerGuru wrote: post #9
RyanMcGreal wrote: post #6

Given the requirement that non-technical users should be able to use it, a wiki seems like a better model: GUI editing and version control.

How well do concepts like forking and branching work with wikis, though? You can create categories and place the pages in them (to put the different editions for each title), but you can't really "link" revisions and branches to one another or indicate where they were derived from. And forking is absolutely out of the question.

It seems that something like GitHub's wikis would be a great compromise: a simple web-based editing interface, but backed by a Git repo that you can also access as a vanilla Git repo with plaintext files.

Re: The Case for a Git-Powered Project Gutenberg

#29

Post by joh6nn »

ComputerGuru wrote: post #9
RyanMcGreal wrote: post #6

Given the requirement that non-technical users should be able to use it, a wiki seems like a better model: GUI editing and version control.

How well do concepts like forking and branching work with wikis, though? You can create categories and place the pages in them (to put the different editions for each title), but you can't really "link" revisions and branches to one another or indicate where they were derived from. And forking is absolutely out of the question.

there are more than a few wikis that use version control for their backends. i'm aware of at least 2 that use git; branching and forking in that case would simply be UI issues, and could be handled the same way that github does.

Re: The Case for a Git-Powered Project Gutenberg

#30

Post by felix_faber »

DanBC wrote: post #18
felix_faber wrote: post #11

Project Gutenberg is a complete mess. It took them years of discussion and they haven't yet chosen a master file format for their books.[1] As a consequence, they are hosting thousands of books in many different file-formats provided by distributed proofreaders[2] and some other sources. But since no master-format exists and conversion is .. well .. buggy, they have no leverage on their quality problems. The textual…

A decent master file system is good. Including conversion is sub-optimal. Have a sister project that creates and maintains good software for conversion. That would draw 2 crowds; people wanting to help PG and people unwilling to continue using Calibre.

IMO, calibre is two-way. From any file format to any other.

That is _hard_ to do really good. Text is the easy part. What about tables? What about image captions and footnotes?

What such a project would need is one master file format from which other formats are derived. But not the other way around.

Having disjoint repositories for books/software is a good idea though.

Post reply on HN