Live data from Hacker News

How I attacked myself using Google and I ramped up a $1000 bandwidth bill

behind-the-enemy-lines.com

121–130 of 152 posts

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#121
post #109

Earlier quoted context omitted.

Except most providers don't charge by time and demand (which would actually make sense), but by a fixed cap or fixed cost per byte. If the problem is congestion as you suggest, surely it would make more sense to charge different amounts based on time of day, length of session, etc? Which would give Amazon more trouble? 1,000 TB spread out over a month, or 1,000 TB spread out over a day? Their current pricing model as…

It is more complicated than that. Customer convenience must be considered. For instance, I and many others like me will not buy data transfer from someone when figuring the cost requires me to account for the time of day and phase of the moon.

Customers already deal with such billing from other companies and it's not difficult.

Ever had a cell phone? During the day counts against an allotment, after a certain time at night, calls are free. For a home internet connection it would probably be reversed, as peak usage is right after work lets out.

Some power companies also do the same thing.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#122
post #90

Earlier quoted context omitted.

I have a VPS on buyvm which includes 2TB/month for $6. Each extra TB is another $2.50. This is not a special case, you can other providers with very good pricing for data transfer. I don't understand how the difference in bandwidth pricing can be so large. I'm also certain that Amazon doesn't pay as much for bandwidth as the guy from buyvm.

buyvm is overselling their bandwidth to you. Real uplink pricing is not 2 dollars per terabyte. If you actually tried to use say 13 terabytes from a single vm, it would work eithe because of rate limmiting on your uplink or they would cut you off.

well, real uplinks are charged on the 95th percentile, mostly. (sometimes capped, and Cogent does 90th percentile)

I'm signing a deal (if all goes well) tomorrow for $0.65 per MB/sec. 5 gigabit commit on a 10GB port, ($1.15/megabit overage charge, billed on the 90th percentile.) From Cogent, probably the cheapest provider that claims 'tier 1' status (there's a lot of argument about Cogent's "tier 1" status... which is kind of funny; If you only have one provider, a good tier 2 is going to be more reliable than a tier 1 anyhow.) but it is a real uplink and I really can use 5 gigabits of that.

If you ran one megabit full out for a month, assuming 2,629,743 seconds in a month and assuming seagate gigabytes, divide by 8 and you get 328717 megabytes. so, uh, yeah, two dollars a terabyte? assuming they are larger than I am, and their BGP mix is the low-end stuff (Cogent and HE.net or the like) they aren't losing money.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#123
post #8

But, why re-downloading every hour? Does merely having the spreadsheet passively open in a browser trigger that, or was some other process re-loading the spreadsheet every hour? (If the former, I wouldn't be as forgiving of Google. I understand the desire not to cache possibly-private data, but proper URL design and conditional GETs should be able to prevent the entire download on an automatic hourly schedule. And ev…

I've seen this behavior with google gadgets as well, even with no user interaction or even having the page open.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#124
post #122

Earlier quoted context omitted.

buyvm is overselling their bandwidth to you. Real uplink pricing is not 2 dollars per terabyte. If you actually tried to use say 13 terabytes from a single vm, it would work eithe because of rate limmiting on your uplink or they would cut you off.

well, real uplinks are charged on the 95th percentile, mostly. (sometimes capped, and Cogent does 90th percentile) I'm signing a deal (if all goes well) tomorrow for $0.65 per MB/sec. 5 gigabit commit on a 10GB port, ($1.15/megabit overage charge, billed on the 90th percentile.) From Cogent, probably the cheapest provider that claims 'tier 1' status (there's a lot of argument about Cogent's "tier 1" status... which i…

Prices have gone down since the last time I had to deal with all of that. I remember the company I was working with paying $1.67 per Mb/sec, I've seen offers for 1 Gigabit/sec for $1000 so roughly $1.00 per Mb/sec (using 1000, not 1024), but I didn't know they would go down to $0.65 per Mb/sec!

Guess the more you commit to, the better pricing you get!

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#125
post #69
post #52

+1 For Amazon for kindly reimbursing the overage charge. -1 For Google for creating what is the biggest threat to content providers by enabling easy-to-use DDOS attacks across the entire interwebs. Seriously, is this what we have to look forward to when Google Spreadsheets, and God knows what else, become ever-more popular? Think about all the additional onerus costs that would be incurred by content providers as mor…

Like the author notes, I don't think the problem was with Google. It was no fault of theirs. The reason: It's a fine line between maintaining privacy policies and managing such events. If google were storing/caching these links, there would have been an outcry from those worried about user privacy and stuff. About the by-the-hour downloads, well there again is a trade-off between providing data quickly and doing a la…

Google are presumably storing/caching these photos for an hour before expiring and redownloading them.

I don't see why Google don't redownload them using If-Modified-Since, and reusing the original downloaded photos in the case that the origin server says they've not changed.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#126

Reminds me of the email cannon we built on Gmail, not exactly on purpose. We needed a full Gmail account to do some testing on our company's email backup system. At the time that meant 8GB of email. And you couldn't just send a bunch of huge attachments, as it had to be like regular email. So we took a gmail account and signed it up for dozens of active linux related email lists (because they were easy to find). The…

Ah yes, the days of computer labs in high school and figuring out that outlook lets you copy and paste emails in the Outbox. Just get a big enough collection and keep copying and pasting. Easy recipe for getting hundreds - thousands of emails.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#127
post #63

Earlier quoted context omitted.

> you'll knock them offline for a given billing cycle Er... Because caps can never be raised mid-cycle?

Sure, they could be, but it's still a pretty blunt alerting system. Depending on the organization, though, you'd have to get purchase approval on the overage, etc. That will depend on specifics. Makes for sticky questions to answer there as well, which is another argument for putting better management tools at the cloud level.

You're making this way more complicated than it needs to be. It's not an alerting system, it's a knob, like a thermostat. I turn the knob where I want it, the system follows the "run until this limit is hit" instruction. If I want to change my mind later, even after the limit is hit and the system shuts down, I can make that decision at that time. I don't want the system or anybody else to make it for me.

As for the "purchase approval" nonsense -- if the knob isn't right for your company, then don't use it, but there are a huge number of companies where the guy turning the knob is the one and only person with any say over money being spent at all.

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#128
post #122

Earlier quoted context omitted.

well, real uplinks are charged on the 95th percentile, mostly. (sometimes capped, and Cogent does 90th percentile) I'm signing a deal (if all goes well) tomorrow for $0.65 per MB/sec. 5 gigabit commit on a 10GB port, ($1.15/megabit overage charge, billed on the 90th percentile.) From Cogent, probably the cheapest provider that claims 'tier 1' status (there's a lot of argument about Cogent's "tier 1" status... which i…

Prices have gone down since the last time I had to deal with all of that. I remember the company I was working with paying $1.67 per Mb/sec, I've seen offers for 1 Gigabit/sec for $1000 so roughly $1.00 per Mb/sec (using 1000, not 1024), but I didn't know they would go down to $0.65 per Mb/sec! Guess the more you commit to, the better pricing you get!

Bandwidth pricing falls dramatically with bulk. Cogent offered me $0.75/meg for a 3g commit, $0.65 for a 5g commit, and $0.50 for a 10G commit. I know a guy that is getting $4.00/meg on a 100M commit on a 1000M pipe, also from Cogent. (and he chose that over going through me for $650/month for a capped full 1000M pipe) Bandwidth is also a "negotiated good" without a standard price, so what I pay may be rather different from what you pay.

On top of that, bandwidth costs fall... dramatically[1] in competitive markets, and I'm in silicon valley, probably one of the most competitive transit markets. According to the graphs I've seen[1] I'm paying below average, but it won't be very long before what I'm paying is average. I was explaining this to the real-estate guy that owns my data center that was wanting to get in on the bandwidth business (he wanted to get people in for cheap and crank up the price on renewal, like you do with real-estate and data-center space) His response? "but then why would anyone want to be in the bandwidth business?"

I mean, the fiber in the ground? that's like real-estate. the prices go up, the prices go down, eh, whatever. But, the amount of traffic you can push over two strands of fiber? that goes up all the time; I have some old cisco 15540 ESPx DWDM units that can do 16 10GB/sec waves over one pair of fiber. They were awesome back in the day. Modern DWDM equipment? you can get 80 100GB/sec waves on one pair of fiber.

It's irritating, though, as like everything in this industry, you have to negotiate for months to get the "real price" - I asked Cogent for a single, capped 1 gigabit port for $1000/month north of 6 months ago. "Call me back when you can get me a buck a meg" They kept calling me back "how about $3 a meg? how about $1.75 a meg?" I mean, even now, they beat the buck a meg price point, but I had to buy a lot more than I needed (I'm splitting it half and half with another company, at cost, so while the transaction cost was huge, once you factor in the discounted setup fees, well, I am still paying more than a grand a month, but it's still a pretty comfortable fee for me.) I imagine Cogent has spent several thousand dollars of salesperson time, and I /know/ I have spend several thousand dollars of my time on this, and they are charging me rather less than if I had wasted almost none of their time. They even dropped the setup charge down to almost nothing.

And now I've gotta do the same thing all over again with a second provider (most likely he.net) What a waste of time and effort all around. I mean, a little bit? it's kinda fun, I mean, sales people are always ridiculously overdressed extroverts, and in this industry, most of them can pick up on my personality and act in a way that is tolerable or even fun for short periods of time, but I really am an introvert. I mean, it can be fun for a while? but man, I have had like 5 meetings the last two days, between dealing with Cogent and dealing with the people that are buying half the pipe from me. It's exhausting, and I guess I have a hard time seeing how this is the most efficient way to sell bandwidth. I mean, I guess some people throw up their hands and pay the $3/meg asking price, and if they can break even on me, god damn, you wouldn't need many of the $3/meg customers to get really rich.

But then that leads the question: why bother with me? I mean, I'm going to turn around and sell transit very near this cost in a very public way, which will mean that more of those $3/meg customers are going to turn around and ask Cogent for a discount.

I guess they can rely on the fact that, well, I'm a scruffy introvert, and no large corporation will do business with me. Still, I mean, Cogent will let me drop 1G (capped, sadly) ports into datacenters where I don't have equipment for $650/month, and I've been running ads saying I would be willing to sell those at cost (and make my profit off of the difference between the list setup fee, $2500, and what they are actually charging me.) - This was mostly a way to get the higher commit pricing without actually paying for it all.

[1]http://drpeering.net/white-papers/Internet-Transit-Pricing-H...

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#129
post #75

Earlier quoted context omitted.

It's not a DDOS becuase it's not distributed and there was no denial of service. And it doesn't work "across the entire interwebs" because the Google bots are rate-limited against most websites. There are some whitelisted sites, like S3, that are not rate-limited. S3 did not go down in this "attack". In fact the app didn't even go down. So the decision not to rate-limit against S3 was sound, since there was no DDOS.

Ironically, it might be DoS in the sense that it drives app maintenance costs so high that the owner could decide to, naturally, deny the service (shut down). Otherwise, maybe it's the new type of flood attack, cost-of-service (CoS), applicable against those who use ‘invincible’ cloud infrastructure such as Amazon's.

Is that a term? "Cost of service attack" ? If not, let's coin it :)

Cost-of-service-attack: Consuming bandwidth or other resources in cloud based solutions to drive up the cost of running the service.

Very easy when the cost is so tightly coupled with the resource use...

Re: How I attacked myself using Google and I ramped up a $1000 bandwidth bill

#130

Earlier quoted context omitted.

I don't understand the privacy concern - a publicly accessible URL doesn't offer any privacy. It's the same as a transparent proxy.

A password works by there is a piece of text that only you know, and thus, when you give that password to a server, that server knows who you are. can't you also view a url as a password? (If only I know the url, then only I can download the file). I am able to give out a url to someone else, so they can access the file, likewise, I can give out my file server's username and password, and whoever has it can also acce…

You have a point, but the difference is that url's are not usually considered secret and are therefore not treated the same way a password would be. For example in the browser..
Post reply on HN