Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google App Engine Silently Stopped Sending Email 5 Weeks Ago

code.google.com

Post reply on HN
Print view View original

81–90 of 119 posts

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#81

Post by awjr »

markdown wrote: post #63
jpatokal wrote: post #58

Google Cloud Support here. To correct the misleading title, this is not a generalized outage of any kind, but an issue with some applications sending mail that trigger spam/malware/etc filters. Since spammers generally do not comply with RFC 3514, filtering is unfortunately not an exact science, so false positives happen and we're working with the customers in question on a case-by-case basis. Also, App Engine is int…

5 weeks ago your infrastructure stopped sending out mail. Our apps didn't change, your infrastructure did. Whether it's your outgoing spam filter that's overzealous, or if a datacenter is on fire is irrelevant. Your infrastructure stopped doing what your docs said it would. Paying customers like myself have apps running on your infrastructure. Your infrastructure stopped delivering the exact same emails that it had b…

I really have to highlight how dangerous an approach this is. You have no idea if the previous user of that IP was spamming. You should seriously consider mailgun or sendgrid. They also bring a lot of features that enable you to track when an email was read etc.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#82

Post by markdown »

cramsdale wrote: post #65

Hey Folks, The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a…

Hi Chris,

> This issue should be limited to new applications where the trust signal might be a bit lower. Thus existing apps / customers shouldn’t be experiencing issues.

1. My app is not new. It's been running without issue for 2 or 3 years.

2. My app is not on the free tier.

3. On average, my app sends out under 8 emails a day, and it's done the same for over 2 years.

How your algorithm considers my app a spam risk sure beats me.

I'm missing 11 days worth of quote requests from customers. Are these recoverable? (is there a hidden outgoing-spam bin?)

My app started sending mail again today after I changed the src of an image in it from https://example.appspot.com/images/logo.png to http://www.example.com/images/logo.png

How well do you think email clients are going to like emails with images embedded in them without HTTPS?

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#83

Post by inopinatus »

jpatokal wrote: post #55
martindelemotte wrote: post #28

I wonder if people at Google know how awful it is to report an Appengine bug. You have to battle for days to get your bug acknowledged. I've noticed that they've added more support people since the early days but it's still a pain as they seem to have an incentive to answer the tickets as fast as possible without doing any real investigation. The linked issue is a good example of this behavior.

Please remember that Google App Engine is entirely free up to (roughly) 5 million page views/month: https://cloud.google.com/appengine/kb/#quota If you're running production services, you should probably sign up for a paid support package, which have guaranteed response times down to 15 minutes: https://cloud.google.com/support/ Disclaimer: I work in Google Cloud Support.

I think this is a terrible attitude. I will never implement it in my own support model. People shouldn't have to pay me in order to report a bug in my system. Arguably I should pay them, because one report is the tip of an iceberg of silent affected customers.

More often than not, what most people want is simply an acknowledgement. They're not seeking a bug bounty. And definitely not the brush-off you just gave the GP: when someone's house is on fire, you don't demand prepayment for the water.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#84

Post by breakingcups »

justinblat wrote: post #66
mattbillenstein wrote: post #19

Don't use appengine for anything -- ever. It's a terrible platform/product and google would rather you move on to GCE anyway...

Greetings folks! One of the App Engine PMs here. That couldn't be further from the truth - we absolutely want people to use App Engine. If you want to manage your own infrastructure, Google Compute Engine is great. If you want something that scales automatically, and requires less management - App Engine is a great option. We've actually made a ton of changes recently, specifically on our support for docker based run…

When will App Engine finally upgrade Go to 1.6? It's currently stuck at 1.4.

Also, why the long delay? Is it because of the switch from C to Go in 1.5?

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#85

Post by boulos »

hartator wrote: post #80
cramsdale wrote: post #68

Earlier quoted context omitted.

I'm sorry, this simply does not make any sense. If you'd take the time to articulate a well-formed case, I will take the time to dig in. Thanks... -- Chris

I think he is saying that GCE being less expensive than App Engine will fore people to migrate.

The new Tesla Model 3 is cheaper than the Model S. Is that Tesla encouraging people to migrate to the Model 3?

They're not the same product, and in this case they serve different points in the space with different underlying requirements. App Engine Standard's cost structure on the backend just hasn't changed nearly as much as compute engine's lately where it's a much more direct: oh look Haswell released, more cores per host at similar dollars yields less $$/core.

Disclosure: I work on Cloud (Compute Engine mostly) and care a lot about our prices.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#86

Post by wastedhours »

brianmcconnell wrote: post #72
cramsdale wrote: post #65

Hey Folks, The issue reported here is linked to App Engine and Gmail tightening up their spam filters. The root cause was an increase in organizations sharding out their spam systems to utilize App Engine’s free tier in such a way that is (a) in direct violation of our ToS and (b) making all of our lives suck a bit more (raise your hand if you want spam). It’s unfortunate that while App Engine is trying to provide a…

Long time App Engine user here. If you're sending email as part of your app, I'd recommend using Mailgun (or similar services). You can set up DKIM, inbound routes, etc, and generally have more control over how your mail is processed. I never got the impression App Engine really wanted to be in the email delivery business, so if that's important to you, there are tools that are focused on that (and free up to a month…

+1 as well, after the changes to Mandrill recently, was looking for a new provider and can also recommend Mailgun (at least, for ease and configurability in set up and testing so far, not yet in production).

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#87

Post by pritambarhate »

wastedhours wrote: post #86
brianmcconnell wrote: post #72

Earlier quoted context omitted.

Long time App Engine user here. If you're sending email as part of your app, I'd recommend using Mailgun (or similar services). You can set up DKIM, inbound routes, etc, and generally have more control over how your mail is processed. I never got the impression App Engine really wanted to be in the email delivery business, so if that's important to you, there are tools that are focused on that (and free up to a month…

+1 as well, after the changes to Mandrill recently, was looking for a new provider and can also recommend Mailgun (at least, for ease and configurability in set up and testing so far, not yet in production).

Yes, Mandrill should have kept the free quota after DKIM and SPF verification. Now, I will be trying out the competitors (Sendgrid and Mailgun) and if I find them at par with Mandrill then for paid projects, I might use them rather than Mandrill.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#88

Post by Confusion »

markdown wrote: post #63
jpatokal wrote: post #58

Google Cloud Support here. To correct the misleading title, this is not a generalized outage of any kind, but an issue with some applications sending mail that trigger spam/malware/etc filters. Since spammers generally do not comply with RFC 3514, filtering is unfortunately not an exact science, so false positives happen and we're working with the customers in question on a case-by-case basis. Also, App Engine is int…

5 weeks ago your infrastructure stopped sending out mail. Our apps didn't change, your infrastructure did. Whether it's your outgoing spam filter that's overzealous, or if a datacenter is on fire is irrelevant. Your infrastructure stopped doing what your docs said it would. Paying customers like myself have apps running on your infrastructure. Your infrastructure stopped delivering the exact same emails that it had b…

Infrastructure always changes. You need to have something in place to detect failure of important features of your apps, that get checked every day, every hour or even every minute, depending on how costly an outage is for you. There are always doublechecks possible.

Re: Google App Engine Silently Stopped Sending Email 5 Weeks Ago

#89

Post by pyvpx »

inopinatus wrote: post #83
jpatokal wrote: post #55

Earlier quoted context omitted.

Please remember that Google App Engine is entirely free up to (roughly) 5 million page views/month: https://cloud.google.com/appengine/kb/#quota If you're running production services, you should probably sign up for a paid support package, which have guaranteed response times down to 15 minutes: https://cloud.google.com/support/ Disclaimer: I work in Google Cloud Support.

I think this is a terrible attitude. I will never implement it in my own support model. People shouldn't have to pay me in order to report a bug in my system. Arguably I should pay them, because one report is the tip of an iceberg of silent affected customers. More often than not, what most people want is simply an acknowledgement. They're not seeking a bug bounty. And definitely not the brush-off you just gave the G…

no, when your house is on fire the property-tax funded fire department comes and attempts to put it out.

the "free tier" of municipal services is actually just a form of socialism!! eek!

Post reply on HN