[deleted]
Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
61–70 of 94 posts
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#62Honestly this feels overly complex when you can just create a CA and add the CA to ur devices. Still cool tho.
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#63Cert-manager has great support for a number of providers[0] including AWS, CloudFlare, Google Cloud, and Azure. I recommend this not just for internal IP setups, for actually for all setups, since DNS verification is more robust than HTTP verification, particularly if you have issues with load balancers, or if Let's Encrypt decides to deprecate a protocol again [1]. [0] https://cert-manager.io/docs/configuration/acme…
Verification via DNS is not without issues. If you have more then one DNS server the verification record need to propagate to all servers. If you for example use anycast DNS you will run into issues. Letsencrypt uses Google name servers for lookup which is problematic because they do not behave, they will for example not try secondary dns servers if the first try fail, making the Letsencrypt verification also fail. A…
i had this issue but i just set the time to wait for propagation high enough to be somewhat certain and had no further issues since (its 10m i think). it does not really matter to me how long it takes as its an automated process... should be finished before expiration though ^^
having multiple nameservers is pretty standard and often mandatory requirement set by the NICs. Its just that my secondary is sometimes not fast enough to transfer the zone after a change notification which triggers this issue.
Also, retrying after an authoritative nameserver said there is defacto no record seems pretty wrong to me... i doubt they use google DNS or anything really. in order to avoid caching issues they very likely resolve names recursively without (the usual) caching
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#64Earlier quoted context omitted.
While you'll get the hostnames leaked - you could register them as fake addresses (say an A record for 192.168.0.1 for every address), and have a local DNS server overriding with the real addresses. Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? The other option for internal certificates is to get a…
> Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? You're probably aware of tools such as dnsdumpster (dnsdumpster.com) that attempt to map internal networks using dns. It is common to use descriptive prefixes (eg. intranet, gw, fw, dns, jira, mysqldb, etc) that also give some insight on the internal t…
Downsides of a single wildcard certificate has its own issues, the more you pass a single certificate round the more opportunity for it to leak and the wider the risk
If you have a per-host DNS based with no valid A records, DOH could well mean worse performance in future as devices don't listen to canary domain and only query locally once getting an NXDOMAIN (and even then might not). You're still leaking the CN and SAN entries via CT.
If you run your own CA, every device has to have it installed, and it's a major security risk as losing control of the CA means all your machines can be compromised.
Between certificate transparency and DNS over HTTPS, there's no perfect answer.
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#65Honestly this feels overly complex when you can just create a CA and add the CA to ur devices. Still cool tho.
At home, with just a few devices, it's doable, but adding a ca on all the devices of a corporate network is a huge pain. On one hand, you have varying levels of control (from none to total) on the devices. On the other hand adding a ca is a bit of a pain, with the ca needed to be added in various ca collection s for different softwares (ex in linux, ca-certificates, java certificates, and to add them system wide for browsers, you need to recompile libnss).
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#66Cert-manager has great support for a number of providers[0] including AWS, CloudFlare, Google Cloud, and Azure. I recommend this not just for internal IP setups, for actually for all setups, since DNS verification is more robust than HTTP verification, particularly if you have issues with load balancers, or if Let's Encrypt decides to deprecate a protocol again [1]. [0] https://cert-manager.io/docs/configuration/acme…
See also lexicon, which supports over 50 APIs:
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#67Cert-manager has great support for a number of providers[0] including AWS, CloudFlare, Google Cloud, and Azure. I recommend this not just for internal IP setups, for actually for all setups, since DNS verification is more robust than HTTP verification, particularly if you have issues with load balancers, or if Let's Encrypt decides to deprecate a protocol again [1]. [0] https://cert-manager.io/docs/configuration/acme…
Verification via DNS is not without issues. If you have more then one DNS server the verification record need to propagate to all servers. If you for example use anycast DNS you will run into issues. Letsencrypt uses Google name servers for lookup which is problematic because they do not behave, they will for example not try secondary dns servers if the first try fail, making the Letsencrypt verification also fail. A…
> If you for example use anycast DNS you will run into issues.
You're not wrong, but this assumes that you use the your 'service hostname' for verification as well, rather than using CNAMEs.
So let us say you want to have "svc1.example.com" in your cert: you could put the ACME challenge under there, but if you have anycast delays that's a problem (as you mention). (A kludge is putting a 'sleep' somewhere to allow for propagation.)
So instead what you can do is have "_acme-challenge.svc1.example.com" be a CNAME that points to (say) "_acme-challenge.svc1.dnsauth.example.com". This sub-domain is not anycast, and may actually be a single machine that is used solely for this purpose.
The LE/ACME server goes to your main domain, finds a CNAME, and follows that to the real record and verification is achieved:
* https://www.eff.org/deeplinks/2018/02/technical-deep-dive-se...
* https://dan.langille.org/2019/02/01/acme-domain-alias-mode/
* https://github.com/acmesh-official/acme.sh/wiki/DNS-alias-mo...
The CNAME has to be set up initially, but can be left lying around otherwise.
This is how $WORK deals with getting LE certs for internal domains: we create a CNAME record (but no A records) for the internal hostname in our external DNS that point to our "dnsauth" domain which gets updating by internal clients via an API.
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#68Honestly this feels overly complex when you can just create a CA and add the CA to ur devices. Still cool tho.
We looked into this at $WORK, but it can be slightly annoying as you have to create a workflow for each operating system's trust store, but you also have to deal with many browsers independently as well, since many of them don't use the OS' trust store.
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#69I just created a wildcard with letsencrypt in the format of .internal.mydomain.com
My public services all run out of .mydomain.com and all my internal services run out of .internal.mydomain.com
I have my internal dns set to resolve any .internal calls to an internal load balancer which hosts the ca certs.
The downside is that all internal services are ssl terminated at the load balancer, but this makes handling internal certs easy as they're rotated in a single location. This is Good Enough for my homelab.
Re: Automatic SSL Certificates for internal IP's for home k8 setup using LetsEncrypt
#70I do this with Traefik [0] internally in almost the same way. I use DNS-01 to get a Let's Encrypt wildcard cert and all my internal A records point to the ingress IP and Traefik happily proxies the communications to the appropriate service - container based and non-container based - which is the real win I was looking to solve for in my home environment. The thing I like about just using Traefik is it doesn't rely on a lot of extraneous tooling (can just use Docker without Swarm/K8s) and will automatically consume orchestration services if I'd like it to. But the reality is the majority of things I want valid certs for are static mappings. One config file update of a few new lines of boilerplate is all it takes to get a valid cert fronting any service. And then to get a dashboard of all my internal services I use Heimdall [1].