Live data from Hacker News

Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

github.com

21–30 of 44 posts

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#21

Somewhat related question: when I go to add a domain to CloudFlare, it did a great job of importing all of my existing DNS records. Since `dig any` doesn't seem to work anymore, how could they be doing this?

They just have a long list of likely host / service names. It misses things all the time for me, but gets an awful lot correct.

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#22

Good news is these tools can stop using the global API token on Cloudflare and use a scoped token just for this purpose: https://blog.cloudflare.com/api-tokens-general-availability/

Indeed, "but" the minimum token you can create is one that allows its wielder to edit a whole zone's DNS, not just (say) one specific record.

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#23
post #19

Earlier quoted context omitted.

It would also probably be good to use Cloudflare's own ip lookup service, available on every CF website's "/cdn-cgi/trace" page. The only thing is that you need to find a ipv4-only CF website to get the v4 instead of v6.

Accessing the cloudflare IP address directly works. For example http://1.1.1.1/cdn-cgi/trace

This is actually the fastest, always responds under 100ms! I put this in, thanks!

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#26

This is not very efficient: - It updates the records every 30 seconds even if it doesn't have to. - It has to be run as a daemon, eating memory and CPU time constantly. - only checks one IP service (ipify) I have a more advanced version, which: - you can run as a cron job - only updates the records if the IP address actually changed. - checks multiple IP services (so in case multiple of them going down, it will still…

It would also probably be good to use Cloudflare's own ip lookup service, available on every CF website's "/cdn-cgi/trace" page. The only thing is that you need to find a ipv4-only CF website to get the v4 instead of v6.

Huh. didn't know this existed! Thanks! Is there a list of other pages cloudflare serves outside of this one?

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#27

This is not very efficient: - It updates the records every 30 seconds even if it doesn't have to. - It has to be run as a daemon, eating memory and CPU time constantly. - only checks one IP service (ipify) I have a more advanced version, which: - you can run as a cron job - only updates the records if the IP address actually changed. - checks multiple IP services (so in case multiple of them going down, it will still…

I like the approach of the program managing its own scheduling much better than cron. You have an endpoint that is constantly available to scrape metrics from. You can send it an RPC to say "hey, do the sync now". The behavior when updating the code is well-defined. This particular daemon doesn't really do any of these things, but when they're needed it's trivial to add.

I also don't think it's a bad idea to do the update at the desired interval. Someone could go into Cloudflare's web interface and mess up the configuration; with this design, the mistake is fixed within 30 seconds. With a design where an update request is only issued if an IP change is detected, then that will probably never be fixed and inconsistent behavior will result. The underlying problem is assuming that your sync program knows the entire state of the Universe and can properly detect a change. It can't and you shouldn't design something that can. All you want to do is push your known-good state to a configuration repository. So that's what the code does.

Strange behavior works when a sync process does different things every iteration. If Cloudflare's API changes for some reason and your program only applies an update when there's a change means you only detect the failure when an IP address change is needed. Now your IP address is wrong in Cloudflare while you rewrite your program. If you just apply the change periodically even though there is no diff, you get a nice alert that the API is broken before you need to change. That increases the probability that you'll have a working program at the time when you actually need to change the IP. Instead of stressing out about the problem, you can take time to fix it properly, and no outage results.

Furthermore, this program properly maintains a global rate limit of API calls across all configurations. That is way easier to do in a daemon (limiter.Take() whenever you feel like making an update) than in a cron job; which is going to have to make a different number of API calls each time it runs, and you don't control when the job runs. The rate limit is likely high enough that you will never notice, but technically this approach is more correct than the "yolo do whatever" cron job.

Programs that are in a sleep(30 seconds) call do not use much memory or CPU. They can be trivially swapped out if the system is under memory pressure. A go application like this is going to use less than 10M of RAM all in. In 1962 that's a lot. Today that's nothing.

Given the existence of a kubernetes configuration in the repository, I think they made the right decision with the design here. Kubernetes does have cron, but cron is a high-overhead thing in Kubernetes. It creates a new job and schedules the associated pod every time the scheduling interval is reached. That's one of the most expensive things you can do, and while no harm will be caused running this every 30 seconds, the maintainability is lower and the probability of something unusual happening is higher. Speaking from experience, "kubectl get events" also becomes completely unusable when you use cron to run frequent jobs.

To be fair, I have no idea what sort of environment involves production traffic in a kubernetes cluster going to a dynamic IP address, and that sounds like something I'd look into before writing a sync script like this. But overall, I think the design is solid. It has the potential to be properly observable. It has consistent behavior. You can detect problems before they become an outage. It properly rate-limits calls to Cloudflare no matter how many records you have configured it to sync. That, to me sounds like a good design.

I would personally have used a statefulset instead of a deployment. A deployment is designed to be 100% available. A statefulset is designed to have the exact number of replicas you specify running at once. For a sync job like this, you probably only ever want 1 running. When you upgrade the software, you don't want both the old version to be syncing stuff while the new version is waiting to pass its first health check. But in this case, I doubt anything bad will happen in that case, so it doesn't really matter.

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#28

Earlier quoted context omitted.

It would also probably be good to use Cloudflare's own ip lookup service, available on every CF website's "/cdn-cgi/trace" page. The only thing is that you need to find a ipv4-only CF website to get the v4 instead of v6.

Huh. didn't know this existed! Thanks! Is there a list of other pages cloudflare serves outside of this one?

Don't think there's a list; the only other pages that I see regularly are for email protection[0], CF apps[1] (the ones that inject scripts), expect-ct (CT monitoring), and image resizing[2].

0: https://blog.cloudflare.com/introducing-scrapeshield-discove...

1: https://www.cloudflare.com/apps/

2: https://blog.cloudflare.com/announcing-cloudflare-image-resi...

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#29

Good news is these tools can stop using the global API token on Cloudflare and use a scoped token just for this purpose: https://blog.cloudflare.com/api-tokens-general-availability/

This is great news! I've always wanted to use the API with fail2ban but didn't want to run the risk of using the global API key.

Re: Cloudflare-sync – Tool for using Cloudflare as a dynamic DNS provider

#30

This is not very efficient: - It updates the records every 30 seconds even if it doesn't have to. - It has to be run as a daemon, eating memory and CPU time constantly. - only checks one IP service (ipify) I have a more advanced version, which: - you can run as a cron job - only updates the records if the IP address actually changed. - checks multiple IP services (so in case multiple of them going down, it will still…

I like the approach of the program managing its own scheduling much better than cron. You have an endpoint that is constantly available to scrape metrics from. You can send it an RPC to say "hey, do the sync now". The behavior when updating the code is well-defined. This particular daemon doesn't really do any of these things, but when they're needed it's trivial to add. I also don't think it's a bad idea to do the u…

[deleted]
Post reply on HN