Live data from Hacker News

GitHub user sends notification to 400k users

github.com

131–140 of 468 posts

Re: GitHub user sends notification to 400k users

#131

Earlier quoted context omitted.

> I try to not mix work and personal content on the same laptop It's kind of jaw dropping to me at how this is still not the norm and how people gratuitously mix personal and work content on the same devices, both mobile and desktop.

I remember when I worked at an Alphabet company and they offered the option of using your personal phone to sign in to work stuff. (It wasn't Slack, it was one of the seventeen different chat systems they had going at the time.) There was a teensy-weensy little caveat: Google IT could wipe your personal device at any time! No thanks.

That’s a widespread problem. I worked for a dramatically smaller IT firm and it also gave employees the opportunity to register their phones with the company’s Exchange service; when I asked my boss whether we should warn them about the power that gave us, he felt it would just cause unnecessary alarm.

Re: GitHub user sends notification to 400k users

#132
post #34

Earlier quoted context omitted.

It doesn’t really fix the “vulnerability” though? Some other jerk can open a new PR and tag everyone again. Or just tag @EpicGames/developers elsewhere, no PR needed.

Looks like they dealt with the tag too as https://github.com/orgs/EpicGames/teams/developers is showing not found

The team is still there, only members can see it.

Re: GitHub user sends notification to 400k users

#133
post #120

I'm confused. Can someone explain why the Epic Games developers team has close to 400k members?

Unreal engine is open source but in a private GitHub repo. Anyone can link their GitHub account with their epic games account which adds them to the team.

Unreal engine is source-available with a royalty based license for commercial use. Just to avoid any confusion with "open source" and "Open Source"

Re: GitHub user sends notification to 400k users

#134
post #14

Earlier quoted context omitted.

There was a thundering herd e-mail at Amazon about 10 years ago that I’ve heard stories about. It went on for days. There’s a funny internal talk with lots of data about it, maybe it’s on YouTube by now…

Something similar happened at Cisco something like maybe 5 years ago? Someone sent the Bay Area employees or whatever list (probably at least some tens of thousands of employees) an ask for a cook/chef they could hire to make meals for their family, IIRC. I think the reply-all's happened all week.

That was sometime in 2015-2016. People still post recipes in big reply-all chains or when cisco-flame gets too heated.

Re: GitHub user sends notification to 400k users

#136
post #62

Earlier quoted context omitted.

friendly reminder: 400k is an apertif for a competently configured production postfix server. its about 14 seconds of mail, and about 8 seconds optimized at hw and filesystem level. the real issue here is shitty projects from shitty companies.

Yes but that’s on top of what GitHub is already sending, plus it must be multiplied by the number of comments left on top of that. It feels like 35 straight minutes of 100% usage isn’t great on any system. It presumably sent 61 million emails.

> Yes but that’s on top of what GitHub is already sending

When you send at the volume someone like GitHub sends, you will always see peaks and valleys in your sending patterns that are much larger than 400k. It might cause an issue if they were already under peak load, but even then it would just take a bit longer.

Re: GitHub user sends notification to 400k users

#137
post #14
post #5

The toothpaste is out of the tube - notifications are going to be scrutinized more in the future, from this point on. I got the email. And about 20+ responses to that email. Every person who replies to that issue triggers another 400k emails. Personally, my email client is crashing. EpicGames, as a GitHub org is an outlier, it's basically an SSO for Unreal Engine. I forgot I was even a member of it.

There was a thundering herd e-mail at Amazon about 10 years ago that I’ve heard stories about. It went on for days. There’s a funny internal talk with lots of data about it, maybe it’s on YouTube by now…

Would that happen to be the "wallet" incident? It was slightly before my time, but I also heard legends of it. During my time there, any email thread that looked headed for another reply-all storm had people replying-all to it with simply the word "wallet", apparently in an attempt to deliberately cause chaos.

Re: GitHub user sends notification to 400k users

#138

Seems like an innocent enough mistake. Remind me of when I once helped organize a volunteer hackathon with people from different companies. We created a Slack organization just for the occasion. At the end of the event, I was supposed to ask all attendees to delete all the data we had given them before they went home. The message I posted to #general (the channel everyone in a Slack organization is required to be in)…

Did the slack warning not tell you the number of people it was going to ping? At least nowadays it says something like "You're going to ping X people in Y timezones".

Re: GitHub user sends notification to 400k users

#139
post #34

Earlier quoted context omitted.

It doesn’t really fix the “vulnerability” though? Some other jerk can open a new PR and tag everyone again. Or just tag @EpicGames/developers elsewhere, no PR needed.

The thing that gets me, as discussed in this thread, is that the solution is often just super easy. Literally just pop up a modal when the recipient list is above 1,000 users that says “You’re about to send this to X,000 users, Are You Sure?” Would have totally avoided this situation.

Adding that check to every incoming comment in a large scale system and adding the previously nonexistent interactive UI for it is far from “super easy”. This is not a Todo MVC. There must be a thousand higher impact stories that are better targets of engineering time (unless this form of trolling become a new pastime).

In addition, there are lots of actually malicious skids you can’t stop by asking nicely.

Re: GitHub user sends notification to 400k users

#140
post #14
post #5

The toothpaste is out of the tube - notifications are going to be scrutinized more in the future, from this point on. I got the email. And about 20+ responses to that email. Every person who replies to that issue triggers another 400k emails. Personally, my email client is crashing. EpicGames, as a GitHub org is an outlier, it's basically an SSO for Unreal Engine. I forgot I was even a member of it.

There was a thundering herd e-mail at Amazon about 10 years ago that I’ve heard stories about. It went on for days. There’s a funny internal talk with lots of data about it, maybe it’s on YouTube by now…

Wallet?
Post reply on HN