Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

arxiv.org

Post reply on HN
Print view View original

91–100 of 204 posts

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#91

Post by roskelld »

prmph wrote: post #74

Why is it so hard to export your own messages out of Discord, Slack, etc? We have regressed from the open email standard and gone back to these opaque islands of data that do not adhere to any standard. Slack refused to show me my own messages past a certain age unless I paid up, and eventually deleted them.

There are tricks to get messages from Slack, though I heard they were changing soon if not already.

A year or so ago I exported all messages from a Slack group I ran and used a Discord bot to recreate the entire dataset including channels and user posts. So we now have our entire history of messages without being blocked by a paywall (Until Discord does the same, and we'll be off to find a new home).

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#93

Post by lolinder »

spencerflem wrote: post #15

insane. people doing awful stuff like this is why the world is retreating into private group chats. these researchers should be ashamed

These researchers are making public scraping that was already happening in private. Do you really think they were the first ones to realize that public Discord servers were public?

On the contrary, kudos to these researchers for bursting the illusion that people previously had that things said on public servers somehow would be ephemeral. That's not how the internet works, that's not how it's ever worked. If you send something to someone else's computer you have always had to assume that every recipient could have made a copy of it, and when the recipient list is "everyone who ever joins a public Discord server from now until the end of time" that makes it public information.

Better that it be widely recognized and talked about as these researchers are doing than have accessing public data remain a dark art that lay people mistakenly believe can't happen.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#94

Post by roskelld »

I don't know if Discord fixed it as I haven't checked in a few years, but I tinkered with scraping some public Discords and I found that I could see hidden channels, not the data, but the channel names, which could do things like reveal to me if the same Discord was used for in-house development if it was a product Discord. Not great.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#95

Post by [deleted] »

GaggiX wrote: post #37
mjr00 wrote: post #31

Earlier quoted context omitted.

> The biggest problem that sucks about discord is that it isn't normally publicly searchable. This is a feature of the platform, not a bug. Because of the lack of discoverability people act more genuine, for better or for worse, than public places like Twitter, Bsky, Facebook, Instagram, etc where you have to maintain your public image and/or act like HR is watching over your shoulder. That being said, this feature a…

>than public places like Twitter That seems to be a counterpoint to your argument. Users on Twitter usually do not hold back.

[deleted]

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#96

Post by zetanor »

charcircuit wrote: post #70

>Data was collected through Discord's public API, adhering to ethical guidelines How is it ethical to break Discord's terms of service? An ethical researcher would respect any contracts that they agreed to and would not violate them to collect more data.

Which ethical system demands that researchers from the DCC/UFMG not breach an unaffiliated commercial ToS during their research?

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#97

Post by halfadot »

AStonesThrow wrote: post #81
01HNNWZ0MV43FF wrote: post #71

Earlier quoted context omitted.

Would the TOS even prevent something like joining a guild, downloading all messages, then leaving?

I'm not sure what you mean by "prevent". A TOS is a legal document designed to put down rules and a legal basis for the service. I don't know what a "guild" is, if it's some Discord thing, and you don't say whether this is a good-faith human who joins, or a bot operator, intending to scrape. The hypothetical is irrelevant here; what is germane is that the expectation of privacy by the individual participants, and the…

>The hypothetical is irrelevant here; what is germane is that the expectation of privacy by the individual participants, and the terms which bind people who use that service.

How can you have an expectation of privacy in a public forum? Where did this bizarre disorder originate, where people knowingly put their writing out there for literally anyone to read, then turn around and start talking about "expectations of privacy" when they realize what it entails?

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#98

Post by AStonesThrow »

Now those of us who've been around the block know that Discord is merely the latest iteration on chat servers such as IRC.

I'm interested to know, from anyone here who's an IRC operator or server/network admin, how the IRC community deals with scraping and bots, because in the early 90s, it was never an issue of corporate Terms of Service or legalese, but typically handled by community standards, and probably, people did whatever they could get away with, and this needed to be anticipated and tolerated by the other participants in any given server or channel.

I doubt that IRC users, back in the day or in the present, have any illusions of privacy, when logging or reflecting or bouncing chats is more or less a built-in feature and an integral component of such a networked chat service.

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#99

Post by DaSHacka »

Davidzheng wrote: post #23
msp26 wrote: post #18

Fantastic. I wonder how many random technical info is buried in these servers. I hate what it's done for game modding.

The algebraic topology server probably contains a huge number of treasures in modern research algebraic topology. I really really hope it's archived in full

Its not difficult to archive yourself, if you really care[0]

I use a dedicated alt account to archive tons of various servers I'm in, and auto-download all attachments. It's nice having regex search capabilities on my local copy of the data too.

[0] https://github.com/Tyrrrz/DiscordChatExporter

Re: Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024)

#100

Post by lolinder »

spencerflem wrote: post #32
d--b wrote: post #29

Earlier quoted context omitted.

People should realize when what they write is public though. The world retreating into private group chats is not a bad thing.

its a shame that we can't get the benefits of both. it sucks having to individually vet everyone one at a time. you miss a lot. but people like these researchers who only want to exploit are making it necessary

These researchers are just using the tools that Discord made available to archive information that was already public. If anyone is to blame for users believing that their messages were private or ephemeral it's Discord, not the researchers.

Look at it like this: There's no chance in hell that intelligence agencies, hacker groups, and whatever other nasties you care to worry about haven't already been using archives just like this for all their nefarious purposes. They just didn't make their usage public because why break the honey pot?

What these researchers did is show what was possible and make their efforts public. Now you are better informed of what was always possible. It's always been necessary to think carefully before putting stuff on the internet, it's just now your bubble is burst.

Post reply on HN