Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bluesky Social Dataset (235M posts from 4M users)

zenodo.org

Post reply on HN
Print view View original

41–46 of 46 posts

Re: Bluesky Social Dataset (235M posts from 4M users)

#42

Post by saithir »

gusfoo wrote: post #39

Meanwhile, over at Blueksy.app a few days ago, the users were incensed about a 1M-post data set and hounded the creator in to withdrawing it. https://bsky.app/profile/danielvanstrien.bsky.social/post/3l...

Because unlike the authors of this set - who went and stripped the posts out of usernames and permalinks to anonymize it - that set you mention just grabbed data out of the API as-is (at least based on its huggingface description that's left over).

That's the difference.

Re: Bluesky Social Dataset (235M posts from 4M users)

#44

Post by spiffytech »

saithir wrote: post #42
gusfoo wrote: post #39

Meanwhile, over at Blueksy.app a few days ago, the users were incensed about a 1M-post data set and hounded the creator in to withdrawing it. https://bsky.app/profile/danielvanstrien.bsky.social/post/3l...

Because unlike the authors of this set - who went and stripped the posts out of usernames and permalinks to anonymize it - that set you mention just grabbed data out of the API as-is (at least based on its huggingface description that's left over). That's the difference.

Just a reminder that anonymization is much harder than merely removing metadata:

Every time I hear "anonymous data", I think of that time AOL published anonymized search logs (for academic research). The anonymization was negligent, and an NYT reporter de-anonymized and tracked down one of the users with the local & personal info present in the search queries.

https://en.wikipedia.org/wiki/AOL_search_log_release

https://web.archive.org/web/20130404175032/http://www.nytime...

Post reply on HN