Live data from Hacker News

Why is my NTP server costing $500 per year? Part 1 (2014)

blog.pivotal.io

71–80 of 113 posts

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#71
post #27

I wonder if they somehow mistakenly joined their server to the region-specific pr.pool.ntp.org group[1]. At the moment, that pool exists, but has no servers in it. So, if you were the only server in the pool, perhaps you would get a lot of Puerto Rican traffic? [1] http://www.pool.ntp.org/zone/pr

Someone else commented in the article, but it's probably related to Puerto Rico ISPs using a NAT because of lack of IPv4 address space. That single ip is probably many many people.

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#73
post #32

Earlier quoted context omitted.

This $500 would have been $60, on a Digital Ocean box. DO has 1 TB/month limit. Their usage was 300 GB/month.

[author] Digital Ocean is a great deal! Thanks for pointing that out. The reason I use {aws,azure,google} to host my NTP servers is that my day job is developing a VM orchestrator (BOSH) for Cloud Foundry, and BOSH doesn't support Digital Ocean yet (AFAIK). But that's a personal choice, and an admittedly expensive one.

What about amazon lightsail?

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#74
post #43
post #39

Earlier quoted context omitted.

Randomly dropping a percentage of requests is stateless.

Is the NTP synchronization done with only a single packet? Because otherwise you'll be interrupting connections constantly.

It is UDP, so it is connectionless.

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#76
post #63

Earlier quoted context omitted.

[author] > I wish he'd explained somewhere how they leapt to examining virtualized NTP clients... I had a hunch [wrongly] that the traffic was caused by a particular operating system. I didn't have enough machines to run the tests on bare-metal, so I virtualized them. And I suspected that virtualization would provide a worst-case scenario (the virtualized clocks would be jittery). My big surprise was that Windows was…

I have never understood why people think kod is a useful setting. Why do you think a misbehaving/improperly configured client is going to honor the kod packet? The kod packet helps with some clients but I have never seen it change the behavior of the most egregious abusers. Just ignore future requests from misbehaving clients, there is not a lot of benefit in saying "please stop misbehaving" to a client that does not…

Can you post a few more lines about this? I'm in the process of standing up a server and would like to know more about controlling the load. Thanks!

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#77

I think this is great look at walking through the analysis. I too experienced a huge spike in NTP traffic in 2014 but it was because of people exploiting NTP for reflection attacks to DDOS other parties. The forced me to use a GPS module and a Beaglebone Black as an internal time server (which has been great)

I have a few questions about that if you have a minute:

What GPS module did you go with and is it still available? Did you have problems getting signal inside (need to be by a window, run an antenna, etc)?

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#78

Earlier quoted context omitted.

[author] Digital Ocean is a great deal! Thanks for pointing that out. The reason I use {aws,azure,google} to host my NTP servers is that my day job is developing a VM orchestrator (BOSH) for Cloud Foundry, and BOSH doesn't support Digital Ocean yet (AFAIK). But that's a personal choice, and an admittedly expensive one.

What about amazon lightsail?

It also runs on AWS so even though they try to trick you into thinking you don't pay for bandwidth you do. Also small EC2 instances are throttled so their server would probably not function most of the time.

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#79
post #32

Earlier quoted context omitted.

This $500 would have been $60, on a Digital Ocean box. DO has 1 TB/month limit. Their usage was 300 GB/month.

[author] Digital Ocean is a great deal! Thanks for pointing that out. The reason I use {aws,azure,google} to host my NTP servers is that my day job is developing a VM orchestrator (BOSH) for Cloud Foundry, and BOSH doesn't support Digital Ocean yet (AFAIK). But that's a personal choice, and an admittedly expensive one.

Funnily enough, last time I had NTP troubles, it was on a BOSH deployment. BOSH wants to use ntpdate to keep time (because real NTP is too normal?) and we'd overzealously configured the security groups around our NAT setup so the machines couldn't reach their NTP server. Whoops.

It'd be nice if BOSH could have detected this problem and warned us about it, ideally during deployment. But it'd be even nicer if we didn't suck at configuring AWS. If you could fix either of those things, that would be great!

Re: Why is my NTP server costing $500 per year? Part 1 (2014)

#80
post #44

Earlier quoted context omitted.

Not really. If an NTP client gets 1/10th of the updates that it wants, it will still keep reasonably good time.

Do NTP clients not retry immediately/very quickly if a request gets lost?

It's more that your device's clocks don't skew that rapidly, unless something is really wrong.
Post reply on HN