Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Governments on GitHub

ui.rilldata.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Show HN: Governments on GitHub

#11

Post by mlinksva »

danthelion wrote: post #8
Dunstark wrote: post #7

Earlier quoted context omitted.

From what I gather from this page it seems organisations need to be added by the community or organisation members. I know a few organisations that aren't on that list for my country like https://github.com/GouvernementFR , https://github.com/DISIC , https://github.com/ANSSI-FR and https://github.com/ansforge . No promises but I might make a PR tonight on the page you mentioned.

That would be great, scraping that site is the entrypoint of my data pipeline!

Just out of curiosity about how people actually access stuff, does scraping mean (as I'd expect I guess) the rendered website, or consuming the data files https://github.com/github/government.github.com/tree/gh-page... ?

Querying wikidata may also be useful https://github.com/github/government.github.com/issues/877

Added: note about your project at https://github.com/github/government.github.com/issues/1167

Re: Show HN: Governments on GitHub

#12

Post by danthelion »

nonameiguess wrote: post #9

No explanation of where this data is coming from or how it's being labeled? Your top table on the right hand side shows seven "countries" but only one of the names in there is actually the name of a country (Norway). It looks like you're just taking the top-level headers from that "who's using Github" page? Civic Hacker is one of those and is showing up as a "country" in your dataset, but clearly isn't. Looking at th…

I can see how the "Country" name was confusing; thank you. Renamed it to "Government" for a bit more clarity.

Re: Show HN: Governments on GitHub

#14

Post by addcommitpush »

I have some doubts about the data: the government of France has supposedly 1 repos, but for instance the French Statistics Office has 244 repos [0], the "digital" team of the Prime Minister's services has at least 537 repos, and that's what's on top of my head.

[0] https://github.com/orgs/InseeFrLab/repositories + https://github.com/orgs/InseeFr/repositories

[1] https://github.com/orgs/etalab/repositories + https://github.com/betagouv

Re: Show HN: Governments on GitHub

#17

Post by danthelion »

mlinksva wrote: post #11
danthelion wrote: post #8

Earlier quoted context omitted.

That would be great, scraping that site is the entrypoint of my data pipeline!

Just out of curiosity about how people actually access stuff, does scraping mean (as I'd expect I guess) the rendered website, or consuming the data files https://github.com/github/government.github.com/tree/gh-page... ? Querying wikidata may also be useful https://github.com/github/government.github.com/issues/877 Added: note about your project at https://github.com/github/government.github.com/issues/1167

I have missed the data file entirely and just extracted all the entities from the rendered website, then enriched all the repositories with some extra info through the API (still ingesting some data for a few repos, hitting the API limit frequently).

The goal is to have a deeper level view of public government orgs, including a more granular view of contributors and how they interact with code.

Great idea about wikidata! That's another data rabbit-hole to climb down into.

Re: Show HN: Governments on GitHub

#18

Post by danthelion »

addcommitpush wrote: post #14

I have some doubts about the data: the government of France has supposedly 1 repos, but for instance the French Statistics Office has 244 repos [0], the "digital" team of the Prime Minister's services has at least 537 repos, and that's what's on top of my head. [0] https://github.com/orgs/InseeFrLab/repositories + https://github.com/orgs/InseeFr/repositories [1] https://github.com/orgs/etalab/repositories + https://g…

Thanks for flagging this! These were missing from the initial dataset I used as a starting point, but I can easily extend it myself.
Post reply on HN