Live data from Hacker News

Google on today’s massive Google+ spam influx: “We ran out of disk space”

venturebeat.com

31–40 of 46 posts

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#31
post #27

Earlier quoted context omitted.

Are you joking?

no, I am pretty serious, down votes dont change reality. there are apps/services which are signing on 99.999% availability. if after having a expertise in managing 200million+ gmail accounts, I would consider it a bad fumble at 5 million+ users

Mentioning devops undermines your credibility. What service is 5 nines? How is that measured?

In general, Google SRE just gets it done. Sometimes people screw up. It happens.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#32

I still have this happen, and every time I tell myself right, today I'm going to put in a warning system that'll stop this happening .... and then I clear up a ton of space and move on. HTTP.sys logging has been a real pain in my ass, every time I deploy a new server I forget to disable it and we do so much traffic it fills the drive completely overnight.

Sounds like you need to automate setting up a new server

That also doesn't quite happen enough to make it a priority!

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#34
post #22

Earlier quoted context omitted.

Even using 3TB consumer grade SATA disks you'd need 20 of them, an enclosure for 20 disks costs a lot more than a couple of grand.

Not really. http://shopping.yahoo.com/721280117-supermicro-sc216-e2-r900...

That's a chassis for 2.5" disks so you'd be looking at 60x1TB disks, and that would mean 3 of those enclosures. Now rack them somewhere and add power - still much more than a couple of grand, we haven't even paid for the disks yet...

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#35
post #22

Earlier quoted context omitted.

Not really. http://shopping.yahoo.com/721280117-supermicro-sc216-e2-r900...

That's a chassis for 2.5" disks so you'd be looking at 60x1TB disks, and that would mean 3 of those enclosures. Now rack them somewhere and add power - still much more than a couple of grand, we haven't even paid for the disks yet...

Yeah, I was a tad hyperbolic in just referring to the disks. I would expect the costs to be around $20/GB/year when you also factor in power - bigger drives are making a difference, but the other factors always cost more than the disks themselves.

It doesn't change the fact that 60TB is tiny for a company whose every product involves storing enormous quantities of data and serving them at monstrous scale.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#36
post #22

Earlier quoted context omitted.

Not really. http://shopping.yahoo.com/721280117-supermicro-sc216-e2-r900...

That's a chassis for 2.5" disks so you'd be looking at 60x1TB disks, and that would mean 3 of those enclosures. Now rack them somewhere and add power - still much more than a couple of grand, we haven't even paid for the disks yet...

Sorry, posted wrong chassis. Look for the SC847, that's 45x 3,5" in 4U. With 32x 2T S-ATA you're looking at roughly $5000 (incl. disks).

And Google likely gets them quite a bit cheaper than that.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#37
post #27

Earlier quoted context omitted.

Are you joking?

no, I am pretty serious, down votes dont change reality. there are apps/services which are signing on 99.999% availability. if after having a expertise in managing 200million+ gmail accounts, I would consider it a bad fumble at 5 million+ users

Oh, I was referring to you asking about so-called "CMM level" when we're talking about Google, not some second-hand suit-run IT company.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#38
post #36

Earlier quoted context omitted.

That's a chassis for 2.5" disks so you'd be looking at 60x1TB disks, and that would mean 3 of those enclosures. Now rack them somewhere and add power - still much more than a couple of grand, we haven't even paid for the disks yet...

Sorry, posted wrong chassis. Look for the SC847, that's 45x 3,5" in 4U. With 32x 2T S-ATA you're looking at roughly $5000 (incl. disks). And Google likely gets them quite a bit cheaper than that.

Eh, I was definitely lowballing a lot with "a couple grand." You also need to factor in power and replication.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#39
post #36

Earlier quoted context omitted.

Sorry, posted wrong chassis. Look for the SC847, that's 45x 3,5" in 4U. With 32x 2T S-ATA you're looking at roughly $5000 (incl. disks). And Google likely gets them quite a bit cheaper than that.

Eh, I was definitely lowballing a lot with "a couple grand." You also need to factor in power and replication.

Not sure what you mean. It's still a couple grand when you factor in power and replication. ;-)

For google it's a rounding error either way. They measure in Petabytes, not Terabytes.

Re: Google on today’s massive Google+ spam influx: “We ran out of disk space”

#40

Earlier quoted context omitted.

That's a chassis for 2.5" disks so you'd be looking at 60x1TB disks, and that would mean 3 of those enclosures. Now rack them somewhere and add power - still much more than a couple of grand, we haven't even paid for the disks yet...

Yeah, I was a tad hyperbolic in just referring to the disks. I would expect the costs to be around $20/GB/year when you also factor in power - bigger drives are making a difference, but the other factors always cost more than the disks themselves. It doesn't change the fact that 60TB is tiny for a company whose every product involves storing enormous quantities of data and serving them at monstrous scale.

Absolutely agree!
Post reply on HN