I think it's a little premature to be calling this a "meltdown". It's like calling one block of houses with power fluctuations a "blackout".
Media outlets are in the business of making sensational claims.
Gmail, Google Maps and YouTube had outage issues
71–80 of 105 posts
Re: Gmail, Google Maps and YouTube had outage issues
#72Our site was using hosted libraries, google fonts, and google analytics. All of which seemed to be behind captchas, throwing CORS errors, and 503ing since this morning. Swapped out JQuery cdn for now.
For something like jquery, you can host locally and fallback to it if the CDN fails. https://stackoverflow.com/a/1014251
Re: Gmail, Google Maps and YouTube had outage issues
#73Earlier quoted context omitted.
I used to have a service that was like that, it was third party and only handled a specific function so I had a cron job that would reboot it at 4am every day. Sometimes the hammer approach is the only solution.
Was the software loading kernel modules? I'm curious why a more simple kill process routine wouldn't work.
Re: Gmail, Google Maps and YouTube had outage issues
#74Earlier quoted context omitted.
2 reasons: 1. If you're still using HTTP 1.x, sharding assets across origins lets the browser load them in parallel (if set up correctly). You can generally load just 6 assets in parallel per origin, and sharding is a way to get around that limit. 2. A library like jQuery is so popular, and is so often served from googles CDN, that chances are a user already has it in their local cache from when they downloaded it on…
> If you're still using HTTP 1.x Not sure why it was phrased that way but...isn't everybody? I know that HTTP/2 is released and browsers support it, but I'm fairly certain that next to nobody is actually doing anything with it.
Re: Gmail, Google Maps and YouTube had outage issues
#75Earlier quoted context omitted.
[Removed]
There is a common practice for Node.js developers to use one of those tools that automatically reboot the site either whenever the single-threaded-event-loop-server crashes, or periodically...
Re: Gmail, Google Maps and YouTube had outage issues
#76Earlier quoted context omitted.
Was the software loading kernel modules? I'm curious why a more simple kill process routine wouldn't work.
Having been in a similar situation, many enterprise software require a precise start order and shutdown, sometime even depending to certain system services or some arcane stuff that then breaks in unpredictable times and often the only sane way to restart the environment is to bring it down whole
The only way to get it into a known good state when it shit the bed was to reboot the machine and then let it sort itself out.
We rapidly moved away from using it.
Funnily enough I'm in a similar situation at the new job with Jasper Reports, god damn if that thing isn't everything bad about Enterprise Java(TM).
Re: Gmail, Google Maps and YouTube had outage issues
#77Earlier quoted context omitted.
2 reasons: 1. If you're still using HTTP 1.x, sharding assets across origins lets the browser load them in parallel (if set up correctly). You can generally load just 6 assets in parallel per origin, and sharding is a way to get around that limit. 2. A library like jQuery is so popular, and is so often served from googles CDN, that chances are a user already has it in their local cache from when they downloaded it on…
> You can generally load just 6 assets in parallel per origin This seems to only apply to Chrome, whereas Firefox will happily download everything as fast as possible. I know this because I fixed a bug recently where chrome was taking so long to download images that other resources on the page were timing out. No problem in Firefox.
[1] http://blog.olamisan.com/max-parallel-http-connections-in-a-...
Re: Gmail, Google Maps and YouTube had outage issues
#78Does not seem to affect Europe. In fact, right now YouTube loads far quicker than it has for the last seven to ten days, where it would take ages to load any YouTube page.
reboot helps. May be Google runs Windows. Then they just need to reboot more frequently, ie. proactively.
Re: Gmail, Google Maps and YouTube had outage issues
#79Earlier quoted context omitted.
For something like jquery, you can host locally and fallback to it if the CDN fails. https://stackoverflow.com/a/1014251
Loading JavaScript libraries synchronously should be avoided if possible, making the above solution not a great one.
Re: Gmail, Google Maps and YouTube had outage issues
#80Earlier quoted context omitted.
Media outlets are in the business of making sensational claims.
Googlegate 2017. Cataclysmic meltdown where I had to wait a whole 10 more seconds for a video to load as it found another source.