Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

arxiv.org

Post reply on HN
Print view View original

81–90 of 204 posts

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#81

Post by AStonesThrow »

01HNNWZ0MV43FF wrote: post #71
AStonesThrow wrote: post #14

It says they used ethical anonymization, but we’ve seen other scrapers are always completely in violation of Discord’s TOS. So did Discord cooperate, or give special authorization for this collection? It wouldn’t appear that they could do so, if privacy belongs to their users at all.

Would the TOS even prevent something like joining a guild, downloading all messages, then leaving?

I'm not sure what you mean by "prevent". A TOS is a legal document designed to put down rules and a legal basis for the service.

I don't know what a "guild" is, if it's some Discord thing, and you don't say whether this is a good-faith human who joins, or a bot operator, intending to scrape. The hypothetical is irrelevant here; what is germane is that the expectation of privacy by the individual participants, and the terms which bind people who use that service.

The TOS clearly didn't prevent the use of API, but it may indeed prohibit such scraping, or threaten repercussions for people who break the terms, especially for someone who republishes the data. Your example of a simple download dump doesn't seem to involve republication, and that seems to be the major issue with scrapers.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#82

Post by lolinder »

ysavir wrote: post #77
pavel_lishin wrote: post #22

Earlier quoted context omitted.

I'm not sure this is awful. These are public discords, no more private than newsgroups, or StackOverflow. Why is it awful to scrape this , but not that ?

A Discover server's status isn't permanent. Any invite-only server can be into a discoverable, public server, and any public server can be taken private. The public/private distinction isn't a permanent reflection of access but is just a reflection of the current permissions. There's no guarantee that users in a public server were in a public server when they joined and engaged.

That sounds like an enormous flaw in Discord's permission model, not something that should be patched over with "shame" on researchers for archiving data that Discord made public.

The fact remains that if you post something in a private discord channel that someone takes public, it's now public information. All these researchers did is expose that fact—the fact that private messages can later be made public is Discord's responsibility and Discord's shame.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#83

Post by ronsor »

Cynddl wrote: post #64
jowea wrote: post #52

Earlier quoted context omitted.

I understand wanting to be careful, but didn't they only grab messages from servers that are already very public? Are Twitter message datasets anonymized?

That's not how GDPR works and in this case the data is clearly anonymised despite the authors' claims. Amongst others, there needs to be mechanisms for users to delete their data, whether it was at some point public or not.

The authors can presumably update the dataset on the site; however, I think past versions remain. Besides that, the GDPR is at odds with the fact that public posts and data almost never goes away. I don't think that reality can be legislated away, try as politicians might.

In all honesty, it's better to reserve the effectiveness for private, personal data, for the sake of practicality.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#84

Post by __loam »

MarcelOlsz wrote: post #76
charcircuit wrote: post #70

>Data was collected through Discord's public API, adhering to ethical guidelines How is it ethical to break Discord's terms of service? An ethical researcher would respect any contracts that they agreed to and would not violate them to collect more data.

edit: Whoops

Awesome analysis dude! I'm sure the judge will love that when discord sues these guys.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#85

Post by __loam »

drooopy wrote: post #19
SirMaster wrote: post #5

The biggest problem that sucks about discord is that it isn't normally publicly searchable. And it seems to be a modern replacement for internet forums which historically were publicly searchable and often had a lot of great information about various hobbies and things.

There is a special place in hell for software (including game) developers who exclusively use Discord to release patch notes, documentation, technical support, etc.

[flagged]

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#86

Post by lolinder »

spencerflem wrote: post #28
pavel_lishin wrote: post #22

Earlier quoted context omitted.

I'm not sure this is awful. These are public discords, no more private than newsgroups, or StackOverflow. Why is it awful to scrape this , but not that ?

they are "open invite" in the sense that its nice to have serendipitous encounters with people on the internet youd otherwise never meet. its for chatting, and the pace of things ensures whatever you say will be buried. the point isn't to make an artifact, like stack overflow. and certainly not be be experimented on.

The pace of things does not ensure that whatever you say remains buried. Discord has a search feature that isn't great but does work. If you want to ensure that what you say is ephemeral you need to use a chat app that has that as a feature, like Signal.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#87

Post by pavel_lishin »

spencerflem wrote: post #28
pavel_lishin wrote: post #22

Earlier quoted context omitted.

I'm not sure this is awful. These are public discords, no more private than newsgroups, or StackOverflow. Why is it awful to scrape this , but not that ?

they are "open invite" in the sense that its nice to have serendipitous encounters with people on the internet youd otherwise never meet. its for chatting, and the pace of things ensures whatever you say will be buried. the point isn't to make an artifact, like stack overflow. and certainly not be be experimented on.

> the pace of things ensures whatever you say will be buried.

It very clearly does not.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#88

Post by vincnetas »

Sanzig wrote: post #25
spencerflem wrote: post #15

insane. people doing awful stuff like this is why the world is retreating into private group chats. these researchers should be ashamed

You'll probably catch a lot of flak for that on HN, but I 100% agree with you. Just because something is public and can be saved for later doesn't mean it should be done so en-masse. These are social spaces where a lot of young people essentially grew up. An important part of social development is making mistakes and learning from them. How can you make mistakes when those mistakes are archived for all time for every…

Instead of "right to be forgotten" i would like to live in a world with the right "to be forgiven". We know that everyone has done stupid shit in the past and some in present. Idea is not to pretend that this didn't happen, but let it go and accept that people can change (grow up) if they demonstrate regret and willingness to change. Side effect is that if they do repeat offences you can draw some conclusions from that instead of just forgetting it.

Let's bring back the concept of reputation.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#89

Post by judge2020 »

01HNNWZ0MV43FF wrote: post #71
AStonesThrow wrote: post #14

It says they used ethical anonymization, but we’ve seen other scrapers are always completely in violation of Discord’s TOS. So did Discord cooperate, or give special authorization for this collection? It wouldn’t appear that they could do so, if privacy belongs to their users at all.

Would the TOS even prevent something like joining a guild, downloading all messages, then leaving?

User bots (including hacked clients) are officially banned by the TOS, which addresses that concern.

The only acceptable API usage is via bots that server owners choose to invite. And while it might be legally OK (if the bot's own TOS says it), I promise no server owner is expecting an invited bot to slurp up every message for use in a data set, whether that be for academic purposes or a potential stalking/"dirt" database.

I highly doubt this is the most ethical instance of data collection.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#90

Post by smileybarry »

bee_rider wrote: post #66
jazzyjackson wrote: post #51

Earlier quoted context omitted.

Discord "servers" are groups you click on an "invite" to join, there is some framing where you are lead to believe the conversation is not public. Bluesky had a similar freakout when HuggingFace packaged up a snapshot of the firehose and this discussion was had ad infinitum - in Bluesky's face every post is explicitly public -- but a very vocal minority of users still felt they had been wronged because no one asked i…

Although, calling it an invite is probably not good or clear communication on Discord’s part. If an invite is publicly posted for anyone to see and there’s no vetting of who is getting the invites… that’s just a public space.

But “publicly posted” discord invites can mean links in streamers profiles, project websites, etc. While anyone can type that website’s URL, it’s generally expected a group of the public would see & follow that invite, e.g. a streamer’s viewers. It’s not a public space like how a shop on the street is a public space.

Plus, even if the server is in Discovery (and thus really publicly advertised), they’re still mostly sorted into 10+ rooms. Just because a library is public, doesn’t mean I should expect Study Room B will be recorded by John Jameson and entered into a public dataset.

Post reply on HN