Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google App Engine Silently Stopped Sending Email 5 Weeks Ago

code.google.com

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#61

Post by Godel_unicode »

user_0001 wrote: post #50
ajackfox wrote: post #37

I'm pretty surprised by the long wait times between "updates" from support... that's pretty rough. I understand it can be hard to get a good signal to noise ratio with support, but when so many other people are reporting issues, perhaps it's time to take it a bit more seriously? This thread is a great example of why support is so hard.

not really. I wish I could get away with providing such crap support as the big players do. If my customers don't get a reply within a few hours, then all hell breaks loose. Even if, there request comes in at 3am / 0300 on a Sunday morning my time. To be fair, when I point this out to them they do apologise, didn't realise time difference, are in a different part of the world where Sunday is not a day off etc. I then…

Oh, is that all? Someone should let Google know.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#62

Post by Godel_unicode »

markdown wrote: post #56
awinter-py wrote: post #43

The problem is that pesky humans can't decide what they want. Not sending mail is a bug but not receiving mail (i.e. spam filtering) is a feature? I can appreciate how this is confusing.

Not sending is a bug when it's undocumented, when it happens without notification, when there's no error message, and when there's no outgoing-spam bin. The GAE docs say "do x and we'll send your mail". If we do x and mail isn't sent, then that is a problem.

Except as soon as you tell the spammer what is blocked and what isn't, they just add that to the playbook.

I agree with you that some measure of notification that there's spam filtering would be good, but it's Google we're talking about. There's spam filtering.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#63

Post by markdown »

jpatokal wrote: post #58

Google Cloud Support here. To correct the misleading title, this is not a generalized outage of any kind, but an issue with some applications sending mail that trigger spam/malware/etc filters. Since spammers generally do not comply with RFC 3514, filtering is unfortunately not an exact science, so false positives happen and we're working with the customers in question on a case-by-case basis. Also, App Engine is int…

5 weeks ago your infrastructure stopped sending out mail. Our apps didn't change, your infrastructure did.

Whether it's your outgoing spam filter that's overzealous, or if a datacenter is on fire is irrelevant. Your infrastructure stopped doing what your docs said it would.

Paying customers like myself have apps running on your infrastructure. Your infrastructure stopped delivering the exact same emails that it had been delivering for years. It did this without notification and without triggering any errors.

Approximately 35 quote requests that should have been delivered have disappeared into thin air. You've cost me money, and I'm disappointed to find out that you'd been notified of this issue weeks before it began affecting me.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#64

Post by an_account »

nikanj wrote: post #54
spicyj wrote: post #41

Earlier quoted context omitted.

They have paid support if you want it.

Paying for the service should be enough to get support.

For a comparision, AWS also makes you pay for prompt support, and all of their services cost money (unlike AppEngine which has a lot of free services).

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#65

Post by cramsdale »

Hey Folks,

The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a free tier that enables developers to easily use our platform, others see it as an opportunity for exploitation. Even more unfortunate is that it has a negative effect on legitimate users. It’s a fine balance that has been highlighted by several users within this thread.

Spam filtering is not a perfect science, and we’re constantly tweaking things -- with our customers in mind. This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues (which was also highlighted by a few within this thread). If this isn’t the case email me: cramsdale@google.com. For those asking, “hey, why am I being penalized for being a new customer?” See my previous comment about spam filtering not being a perfect science. Then email me.

We’re here and we want to help.

-- Chris (Lead PM for App Engine)

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#66

Post by justinblat »

mattbillenstein wrote: post #19

Don't use appengine for anything -- ever. It's a terrible platform/product and google would rather you move on to GCE anyway...

Greetings folks! One of the App Engine PMs here. That couldn't be further from the truth - we absolutely want people to use App Engine. If you want to manage your own infrastructure, Google Compute Engine is great. If you want something that scales automatically, and requires less management - App Engine is a great option. We've actually made a ton of changes recently, specifically on our support for docker based runtimes, new docs, and new runtime support (like nodejs, ruby, python3, etc).

https://cloud.google.com/nodejs https://cloud.google.com/ruby https://cloud.google.com/python .... etc

If you have any questions at all about the service, feel free to ask me anything.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#67

Post by je42 »

cramsdale wrote: post #65

Hey Folks, The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a…

Isn't there an API to find out why a message was SPAM filtered ?

Or are you afraid that "rogue" applications will use it to produce messages that are SPAM but yet not trigger the SPAM filter ?

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#68

Post by cramsdale »

groks wrote: post #31
enneff wrote: post #21

Earlier quoted context omitted.

> google would rather you move on to GCE anyway... That's not true. We continue to invest heavily in App Engine as a platform.

Appengine's whack pricing says: we'd like to kick you off appengine onto GCE but that would scare away all the enterprise users from GCE. If we announce regular Moore's Law following price cuts for GCE and nothing for appengine, well hey, we can't stop you leaving. We're totally not twisting you're arm! (aside: we are) "A commitment to Moore's Law: https://cloud.google.com/pricing/philosophy/ "

I'm sorry, this simply does not make any sense. If you'd take the time to articulate a well-formed case, I will take the time to dig in. Thanks...

-- Chris

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#69

Post by tacon »

djsumdog wrote: post #46
achant wrote: post #16

Earlier quoted context omitted.

And only sometimes. The issue resolved itself on my app two weeks ago. My guess is there is an outgoing spam filter run amok in some of the clusters.

I wrote about this last year, but I feel it's still relevant. Spam filters, especially Google and Microsoft's (whose e-mail servers don't play by any of the standard rules nor do they send DMARC reports), are horrible over aggressive. http://penguindreams.org/blog/how-google-and-microsoft-made-... I have seen more phishing/ransomware e-mails come through mine (lots of malicious Javascript made to look like Excel file…

>especially Google and Microsoft's (whose e-mail servers don't play by any of the standard rules nor do they send DMARC reports)

I don't know about Microsoft, but I get a couple of DMARC reports per day from Google's email infrastructure. I also get reports from aol.com, yahoo.com, linkedin.com, comcast.com, fastmail.com, etc.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#70

Post by fpgaminer »

cramsdale wrote: post #65

Hey Folks, The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a…

Just a quick thank you for the free tier that App Engine offers. I use it to provide an educational app for my wife, who's a teacher, that she uses as part of her classes. It's nice to be able to provide that to her and her class, without hitting the bank (the school does that enough...). So, thank you.
Post reply on HN