Live data from Hacker News

Ask HN: Google Search down?

news.ycombinator.com

191–200 of 286 posts

Re: Ask HN: Google Search down?

#191

Earlier quoted context omitted.

They kind of do store the internet though, don't they? They store a cached version of most pages.

Tbh the source code of the whole internet would probably be a few PB at most, text is really cheap to store especially because it can be compressed. Images and videos are what makes the premise impossible because even with perfect compression you need an impossible amount of storage to store every video published by mankind.

Well, most of that is on youtube, so they kinda have a copy anyway :)

Re: Ask HN: Google Search down?

#193
post #140

https://www.sfgate.com/news/article/google-electrical-incide... Three people in critical condition after Google data center 'electrical incident' in the Council Bluffs, Iowa datacenter (us-central region, I believe)

Seems unrelated. This happened ~10 hours ago.

Could be related if eng tried to route traffic away from Council Bluffs after the accident (shut down data center for safety inspection/repair?) and failed.

Re: Ask HN: Google Search down?

#194
post #170

looks like leetcode interviews don’t work for finding the right talent with actual skills and experience

I just spent the last 7 years of my life memorizing every problem on leetcode and codeforces using different pattern memorization techniques in order to prepare for my entry level software engineering job at Spotify and now you're telling me this? Absolutely riddiculous.

Re: Ask HN: Google Search down?

#195

No Worries! The Google Server in Mountain View has had a disk failed in it's RAID and therefore the sqlite-DB has only half the iops to it's disposal. A new 500GB hard disk is already in dispatch with Amazon. These are the moments why Google is a proud Amazon Prime customer.

I know many people’s immediate reaction is that 500GB seems small, but remember, they’re not storing the whole internet — just an index of it. The whole internet would need a much larger RAID, with literally dozens of hard disks.

To be fair, this is not actually all that far off the truth. Do napkin math on how much storage you'd need to index say 25 billion documents, and you'll see it's not actually a lot. 25 billion bytes is 25 Gb. How many bytes do you need per document (for the index)? A kilobyte, maybe? If so that's 25 Tb. While certainly not something you can fit on a thumbdrive, it's hardly something you need a planetscale computer cluster to store.

You may need it for the IOPS of actually using it, but that's another thing entirely.

Re: Ask HN: Google Search down?

#196
post #140

https://www.sfgate.com/news/article/google-electrical-incide... Three people in critical condition after Google data center 'electrical incident' in the Council Bluffs, Iowa datacenter (us-central region, I believe)

An arc blast isn’t anything to play with! It’s an electrically charged fireball that is 4 times hotter than the sun at the point of the arc. Pretty much instant 3rd degree burns, while simultaneously getting zapped and hit with an explosive shockwave. https://youtu.be/-iClXrd50Z8 https://youtu.be/PO6see7_ODY

The physics of those things is scary.

If I recall correctly, the mitigating PPE for expecting an arc flash is reflective, because so much of the energy imparted on a victim is straight-up electromagnetic radiation. You get lightbulbed to death.

Re: Ask HN: Google Search down?

#197

Earlier quoted context omitted.

I know many people’s immediate reaction is that 500GB seems small, but remember, they’re not storing the whole internet — just an index of it. The whole internet would need a much larger RAID, with literally dozens of hard disks.

They kind of do store the internet though, don't they? They store a cached version of most pages.

Should be noted HTML compresses extremely well.

Re: Ask HN: Google Search down?

#198
post #111

https://www.cnn.com/2009/TECH/06/26/michael.jackson.internet... This reminds me of the distant past. The death of Michael Jackson resulted in ( almost literally) the entire internet suffering an outage.

More than that on 9/11. Every news site, online forum any online resource for news you can think of was unaccessible for hours. TV turned out to be the only news source.

Cell / landlines were dicey too.

Re: Ask HN: Google Search down?

#199

Really curious to see how a single (albeit large) DC issue caused a global outage. Further; was the DC not protected by redundant power? Also even more interested to see how this arc flash even happened. Presumably these were experienced engineers working with obviously dangerous power levels.

> how a single (albeit large)

That's the reason. In principle, Google distributes execution and data storage for its mission critical applications. And most of the time, they do a pretty good job of that.

But (especially with the hardware crunch in recent years brought about by various effects, including COVID and Bitcoin mining putting pressure on the market) their ability to build out new datacenter capacity has stalled, and as a result, things sometimes end up bunched up in suboptimal ways, and a high capacity data center can become a single point of failure even though it shouldn't be.

> Was the DC not protected by redundant power?

Depending on where the flash occured, it may have been at the point the redundancies feed into the building.

Post reply on HN