Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pplapi: A virtual database of the entire human population

pplapi.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Pplapi: A virtual database of the entire human population

#12

Post by pdabbadabba »

idm wrote: post #9
pdabbadabba wrote: post #7

Earlier quoted context omitted.

This is really fascinating. Have you given any thought to the possibility of providing this synthetic population on a finer grained basis? It could be extremely useful if this were available for, e.g., each U.S. county (or even something smaller than that). Edit: I noticed that each person has a lat/long location. So maybe a different way of framing my question would be, at what level of geographic granularity does t…

Absolutely. This is an active project and I have a plugin system for synthesizing additional data sources. In the research literature, I have seen several impressive US census-derived synthetic population projects, so finer-grained data are a future direction.

Great! I've seen the U.S. Census Bureau's American Communities Survey, which has extremely fine grained data, but with a few gaps for, e.g., religion, Internet access, etc. Do you know of anything else as granular as ACS?

This, again, is really really interesting. When it comes to the fine-grained U.S. data, we may be approaching if-you-dont-build-it-I-will territory. :)

Re: Pplapi: A virtual database of the entire human population

#13

Post by dhimes »

idm wrote: post #8
ReedJessen wrote: post #3

This is fascinating. It's not quite clear to me however if these are real people who have been anonymized or if they are a unitary statistical representation of the population? Let's say I was looking for a startup opportunity. What kind of products can I build with this?

They are not real people, but it is not a "unitary statistical representation" either. At the country level, crossed by the age/sex level, pplapi agents track pretty closely to real humans. Other dimensions (e.g. income) have more noise in them, so they are less reliable.

That explains why a 3 year old in Egypt had an income of $2500 USD or so. I was wondering if it included family income or something else not obvious from a cursory glance.

Re: Pplapi: A virtual database of the entire human population

#15

Post by ideonexus »

idm wrote: post #6
divbit wrote: post #4

Earlier quoted context omitted.

Typically, when one does a university research study in the US involving human subjects there are something like IRB requirements (e.g. [ http://humansubjects.stanford.edu/new/resources/consent/ ] I see research mentioned on the website, and also that the entire world population is mentioned in the data set. Have you run into anything like this?

It's a synthetic world population derived from public population data; these are not real people (i.e. human subjects) and the source raw data were not identifiable or even individual-level data. I nevertheless think there are a number of issues raised by the possibility of a human population database, but to be clear: pplapi does not have actual people in it. I am currently writing about some of these issues.

Until I looked at the actual data, I didn't realize this was synthetic either. The developer might want to make that clear in the header text. "Virtual Database" doesn't make this clear. I think "virtual database synthesizing the entire human population" might make it clearer.

That said. This is very impressive and has my imagination tingling with ways to apply it.

Re: Pplapi: A virtual database of the entire human population

#17

Post by 24gttghh »

dom0 wrote: post #14

DOB: 2012-09-09 Age: 5 Language: Russian Religion: Muslim 10-15% Income: $18466 USD Hmmm

I'm sure this type of data point exists in real life in terms of identity theft schemes...

edit: from browsing more random "agents", there are a lot of 5 y/o's making tens of thousands of $ per year:

Location Country: United States GPS: (36.073868, -103.923638) Demographics Sex: Male DOB: 2012-04-11 Age: 5 Language: English Religion: Protestant Income: $57834 USD Internet: True

Ahh, to be young.

Re: Pplapi: A virtual database of the entire human population

#18

Post by guessmyname »

The limit seems to be 2-7171922938 [1][2] for some reason 1 throws an error [3].

Edit: I should have read the documentation first [4].

> The current database contains 7,171,922,938 agents and is approximately 6.8 TB in size.

[1] http://pplapi.com/2.html

[2] http://pplapi.com/7171922938.html

[3] http://pplapi.com/1.html

[4] http://pplapi.com/docs/

Post reply on HN