Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The entire US Code is now online in XML

uscodebeta.house.gov

Post reply on HN
Print view View original

101–110 of 244 posts

Re: The entire US Code is now online in XML

#101

Post by mjn »

DannyBee wrote: post #77
sc68cal wrote: post #30

Earlier quoted context omitted.

This is currently done via scraping: https://github.com/divegeek/uscode The diffs are huge.

Remember that diff is an algorithm to generate the smallest set of operations to produce version B from version A, not an accurate reconstruction of what happened. Diff algorithms are also often tuned not try as hard to find the smallest set of changes for larger documents, due to speed concerns.

Git's built-in diff algorithm is particularly bad for text. Since it's aimed at line-oriented code, it does line-based diffs, which is horrible for ASCII text that is reflowed, because every line in a paragraph will show up as changed for a small change.

Example: https://github.com/divegeek/uscode/commit/1fb2d83137dad1c6ca...

What's happened is that "Section 2" was moved later in the sentence, abbreviated as "Sec. 2", "of" was deleted, and "act" was capitalized:

    Section 2 of act July 30, 1947, ch. 392, 61 Stat. 674, provided...

    Act July 30, 1947, ch. 392, Sec. 2, 61 Stat. 674, provided...
The rest of the paragraph is unchanged, but git shows a 6-line diff with the entire paragraph replaced. GitHub attempts to do some word-based highlighting (see the timestamp lines), but it falls down on most of these paragraphs. Wikipedia's diffing tends to work better for this kind of thing; I'm not sure what they use. The upshot is that the number of lines changed may be a 5-10x overestimate.

Re: The entire US Code is now online in XML

#103

Post by anigbrowl »

OldSchool wrote: post #75
OldSchool wrote: post #53

Caveat for for many of us overly rational thinkers: the powers that be deliberately are allowed to 'interpret' this code nondeterministically by many different means including its 'spirit,' admissibility of relevant information, manipulation of venue and participants, apparently even extrajudicial proceedings lately. In short, that allows a lawyer to answer almost any question with "it depends," and start billing.

At least for me, accustomed to logic, determinism and mathematical models for laws of nature, this was actually a surprise. Instinctively I still find it distasteful, though I can't say I have anything better to offer other than the unrealistic ideal that all judges would be guaranteed to be truly impartial. I have never directly benefited nor been harmed by the nature of this 'code' either, unless you count getting…

http://en.wikipedia.org/wiki/G%C3%B6del's_incompleteness_the...

Re: The entire US Code is now online in XML

#104

Post by mpyne »

GFischer wrote: post #88
mindcrime wrote: post #54

Earlier quoted context omitted.

In practice, it probably doesn't make any difference. Assuming, for the sake of argument, that XSD is expressive enough to allow the authors to say what they're trying to say, in an unambiguous way, it's more or less a moot point. RELAX/NG is awesome, sure... but to some extent the "war" between it and XSD is a religious war. Pretty much every popular & widely used language / platform has XSD support... hell, it's so…

I didn't need to know about INTERCAL and Befunge :) https://en.wikipedia.org/wiki/INTERCAL http://en.wikipedia.org/wiki/Befunge

It could be worse... lookup the Piet programming language (be sure to include all 3 words in the search) :).

Re: The entire US Code is now online in XML

#105

Post by jamieb »

lobotryas wrote: post #57
OldSchool wrote: post #53

Caveat for for many of us overly rational thinkers: the powers that be deliberately are allowed to 'interpret' this code nondeterministically by many different means including its 'spirit,' admissibility of relevant information, manipulation of venue and participants, apparently even extrajudicial proceedings lately. In short, that allows a lawyer to answer almost any question with "it depends," and start billing.

You make this sound like a bad thing. Would you really prefer "black and white" laws that leave zero room for flexibility or interpretation in light of a given situation?

Yes, we should have black and white laws, and unit tests for them. So someone could write a unit test for "Will this allow the NSA to create a secret court that is outside the rule of law?" or "Does accessing a university computer that has an open access policy allow for a sentence of 130 years?"

I believe we'd see a lot simpler laws.

Re: The entire US Code is now online in XML

#106

Post by mpyne »

drv wrote: post #98
mpyne wrote: post #41

This is awesome. They even have a stylesheet apparently. However though the file claims to be UTF-8, vim seems to disagree, at least for title 10. I can't tell what encoding it really is though, doesn't seem to be latin1 or windows-1252 either.

The encoding looks like valid UTF-8, at least for the first few pages that I glanced at. I did notice the section references look a little strange in vim, e.g. "act Aug. 10, 1956, ch. 1041, § 1" near the top; it consists of c2 a7 (section sign), which looks fine, followed by e2 80 af (narrow no-break space), which shows up as a box in vim.

OK, maybe it's just a font thing here on Windows (wouldn't surprise me one bit). I'll try again when I get home tonight.

Re: The entire US Code is now online in XML

#107

Post by mpyne »

liscovich wrote: post #100
krob wrote: post #97

does that mean these are all the current federal laws?..

This is only the codified portion of the federal law. So called "Statues at Large" that are passed by Congress but not codified into USC are not included. Plus this does not include the federal common law based on the rulings of federal courts.

Also there is the Code of Federal Regulations to contend with, but those are available online already.

Re: The entire US Code is now online in XML

#108

Post by mkehrt »

markdown wrote: post #96
pnathan wrote: post #6

I'm really tempted to collect the XML files and put them on github, with periodic checkpoints to update it with the latest. Watching the evolution of law over time is a fascinating thing and using SW engineering tools to help would be really fun.

NZ laws on github: https://github.com/Br3nda/legislation/tree/master/act/public

Do they accept pull requests?

Re: The entire US Code is now online in XML

#109

Post by gmartres »

mjn wrote: post #101
DannyBee wrote: post #77

Earlier quoted context omitted.

Remember that diff is an algorithm to generate the smallest set of operations to produce version B from version A, not an accurate reconstruction of what happened. Diff algorithms are also often tuned not try as hard to find the smallest set of changes for larger documents, due to speed concerns.

Git's built-in diff algorithm is particularly bad for text. Since it's aimed at line-oriented code, it does line-based diffs, which is horrible for ASCII text that is reflowed, because every line in a paragraph will show up as changed for a small change. Example: https://github.com/divegeek/uscode/commit/1fb2d83137dad1c6ca... What's happened is that "Section 2" was moved later in the sentence, abbreviated as "Sec. 2"…

> Since it's aimed at line-oriented code, it does line-based diffs

You can do word diffs with git:

    git diff --word-diff=color

Re: The entire US Code is now online in XML

#110

Post by ChuckMcM »

DannyBee wrote: post #63
ChuckMcM wrote: post #36

This is pretty awesome, and if it were in git/hg would have the ability to write a 'blame' tool to figure out who voted on the part of the law that is pissing you off :-)

Actually, you wouldn't. This is the US code, not the legislative info :) I actually tried to create this once (with a team behind me, in fact) with what's available or possible with the legislative info. THOMAS theoretically published in XML, but it's missing a lot of info. Not only is this info not published, it's not even stored. They still are literally passing bills around to each other in some cases. You'd have…

Fascinating. Ok that is disappointing but understandable I guess.
Post reply on HN