Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft failed to rotate certificates for winget CDN on time

cdn.winget.microsoft.com

Post reply on HN
Print view View original

11–20 of 69 posts

Re: Microsoft failed to rotate certificates for winget CDN on time

#11

Post by 8organicbits »

TedDoesntTalk wrote: post #8
vbezhenar wrote: post #3

While I appreciate TLS, this thing with certificate expiration is one of the biggest sources of downtime IMO. Something should be done about it. May be throw error not permanently but in a some probabilistic way. Like if 1 year certificate expired, after 3 months 25% of connections would fail. It'll allow eventually to find out about problem but it'll allow for connections to somewhat work, with few retries here an t…

We already have CRL lists. So Why do we need certificate expiration?

When a certificate expires, it can be removed from the CRL. If certificates never expire then the CRL grows without bound.

Also, checking CRL is implemented in different ways. Some checks may be "soft", where a connection failure to the CRL is ignored. You probably want this anyway, if the CRL goes offline you don't want the internet to break. An expiry check, on the other hand, works as long as your clock is accurate.

Re: Microsoft failed to rotate certificates for winget CDN on time

#12

Post by WirelessGigabit »

thayne wrote: post #9
josephg wrote: post #4

I think the default certificate expiration time (2 years) is a terrible idea. Its long enough that there's a good chance whoever registered the cert last time has left the team or the company. Its long enough that I've forgotten how to generate a certificate with openssl on the command line. And its long enough that each time, I (and everyone else) can justify not bothering to automate the process. But 2 years is sti…

> And its long enough that each time, I (and everyone else) can justify not bothering to automate the process And even worse, if you do automate it there is a pretty good chance something changes and breaks your automation by the time it is needed. And that is assuming you actually tested the automation before your new cert is close to expiring.

I solve this by certificate expiration monitoring and renewing the certificate at the 60 day mark.

The expiration warning is configured so that it starts to yell at me if it passes that timeframe.

That gives me plenty of time to fix it IF it goes wrong.

Re: Microsoft failed to rotate certificates for winget CDN on time

#13

Post by moremetadata »

8organicbits wrote: post #11
TedDoesntTalk wrote: post #8

Earlier quoted context omitted.

We already have CRL lists. So Why do we need certificate expiration?

When a certificate expires, it can be removed from the CRL. If certificates never expire then the CRL grows without bound. Also, checking CRL is implemented in different ways. Some checks may be "soft", where a connection failure to the CRL is ignored. You probably want this anyway, if the CRL goes offline you don't want the internet to break. An expiry check, on the other hand, works as long as your clock is accurat…

You are forgetting something.

If you want to reinstall some old software, lets say MS Small Business Server 2000 or Small Business Server 2003 today, the certificates in the installation files prevent the installation of said software. So you wouldnt even get as far as being able to remove any certs.

Your only recourse is change the system date and time back to before the certificates in installation files would have expired.

Besides being a stealth way to prevent old software from being reinstalled, it narrows down the window of opportunity for hackers.

I used to automatically issue certs for my own servers which lasted 24hr's because if a hacker had got in to my system without me knowing which is a real possibility, at least an expired cert being used by someone else would highlight this problem.

As it happened, despite locking everything down to packet level and controlling the packets, my devices were just prevented from getting online. My ISP at the time TalkTalk had a very responsive system, issuing new IP address every 2 seconds in a bid to prevent me from hosting a website, with a domain name using dynamic ip address domain name service.

There is way more surveillance than most people realise at least here in the UK.

Re: Microsoft failed to rotate certificates for winget CDN on time

#14

Post by thayne »

TedDoesntTalk wrote: post #8
vbezhenar wrote: post #3

While I appreciate TLS, this thing with certificate expiration is one of the biggest sources of downtime IMO. Something should be done about it. May be throw error not permanently but in a some probabilistic way. Like if 1 year certificate expired, after 3 months 25% of connections would fail. It'll allow eventually to find out about problem but it'll allow for connections to somewhat work, with few retries here an t…

We already have CRL lists. So Why do we need certificate expiration?

CRL lists have a lot of problems. The biggest being that they are, well big.

Re: Microsoft failed to rotate certificates for winget CDN on time

#15

Post by thayne »

phpisthebest wrote: post #6
josephg wrote: post #4

I think the default certificate expiration time (2 years) is a terrible idea. Its long enough that there's a good chance whoever registered the cert last time has left the team or the company. Its long enough that I've forgotten how to generate a certificate with openssl on the command line. And its long enough that each time, I (and everyone else) can justify not bothering to automate the process. But 2 years is sti…

>>to set up processes which automate renewal. that is all fine and good for things that have the ability to automate that process, plenty of hardware and device do not. Some are not even legacy are still actively being sold and developed It is also not good for internal networks where you can not valid out to something like lets encrypt to automate that validation process, sure you could do your own internal PKI and…

> It is also not good for internal networks where you can not valid out to something like lets encrypt to automate that validation process, sure you could do your own internal PKI and run your own CA for that but......

Or you can set up certbot or similar on a public facing server (or something that can add DNS records to for your domain), and use a secure channel to send the private keys to the things that need it.

I would like to see more of a push to make setting up an internal CA a lot easier though. Because that is probably most correct way to handle that.

Re: Microsoft failed to rotate certificates for winget CDN on time

#17

Post by t0mas88 »

moremetadata wrote: post #13
8organicbits wrote: post #11

Earlier quoted context omitted.

When a certificate expires, it can be removed from the CRL. If certificates never expire then the CRL grows without bound. Also, checking CRL is implemented in different ways. Some checks may be "soft", where a connection failure to the CRL is ignored. You probably want this anyway, if the CRL goes offline you don't want the internet to break. An expiry check, on the other hand, works as long as your clock is accurat…

You are forgetting something. If you want to reinstall some old software, lets say MS Small Business Server 2000 or Small Business Server 2003 today, the certificates in the installation files prevent the installation of said software. So you wouldnt even get as far as being able to remove any certs. Your only recourse is change the system date and time back to before the certificates in installation files would have…

What? 2 seconds? That sounds crazy. It either breaks ongoing connections or wastes addresses since you would have to keep your old one until all connections are closed.

Re: Microsoft failed to rotate certificates for winget CDN on time

#18

Post by Hamuko »

phpisthebest wrote: post #6
josephg wrote: post #4

I think the default certificate expiration time (2 years) is a terrible idea. Its long enough that there's a good chance whoever registered the cert last time has left the team or the company. Its long enough that I've forgotten how to generate a certificate with openssl on the command line. And its long enough that each time, I (and everyone else) can justify not bothering to automate the process. But 2 years is sti…

>>to set up processes which automate renewal. that is all fine and good for things that have the ability to automate that process, plenty of hardware and device do not. Some are not even legacy are still actively being sold and developed It is also not good for internal networks where you can not valid out to something like lets encrypt to automate that validation process, sure you could do your own internal PKI and…

>It is also not good for internal networks where you can not valid out to something like lets encrypt to automate that validation process

Why not? Just use DNS validation.

Re: Microsoft failed to rotate certificates for winget CDN on time

#19

Post by derefr »

WirelessGigabit wrote: post #12
thayne wrote: post #9

Earlier quoted context omitted.

> And its long enough that each time, I (and everyone else) can justify not bothering to automate the process And even worse, if you do automate it there is a pretty good chance something changes and breaks your automation by the time it is needed. And that is assuming you actually tested the automation before your new cert is close to expiring.

I solve this by certificate expiration monitoring and renewing the certificate at the 60 day mark. The expiration warning is configured so that it starts to yell at me if it passes that timeframe. That gives me plenty of time to fix it IF it goes wrong.

In your case, "something breaks in your automation" might mean that, by the time the cert is (about to be) in need of renewal, the notifications you set up are now going to an email account that doesn't exist any more, because you left the department got re-orged and...

Re: Microsoft failed to rotate certificates for winget CDN on time

#20

Post by josephg »

Uvix wrote: post #5
josephg wrote: post #4

I think the default certificate expiration time (2 years) is a terrible idea. Its long enough that there's a good chance whoever registered the cert last time has left the team or the company. Its long enough that I've forgotten how to generate a certificate with openssl on the command line. And its long enough that each time, I (and everyone else) can justify not bothering to automate the process. But 2 years is sti…

The maximum expiration time is now down to 13 months, for certs that need to be valid in a browser. And if you want to cycle yours more frequently, you can. But there's enough places that can't set up automated processes that trying to make it 90 days for everyone would be a lot of pain and a lot of broken sites.

> But there's enough places that can't set up automated processes

Why can't they be automated?

And anyway, this is the exact problem that short expiration times avoid! Systems that aren't set up for automation, and rely on someone once a year remembering some creaky, error prone process to get a new cert. Much better to force short expiration times so manual cert renewal is a thing of the past.

Post reply on HN