Live data from Hacker News

Every day at the same time, my internet dies for 1 minute. How do I investigate?

news.ycombinator.com

61–70 of 141 posts

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#61
post #59

Earlier quoted context omitted.

Actually it seems that it is not renegotiating everyday: it is expiring and re-requesting. The way DHCP 'should' work is that when a client is part-way through the lease it should try to renew the IP it already has: * https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Pro... * https://tools.ietf.org/html/rfc2131#section-4.4.5 This allows the client to (hopefully) keep extending the IP it has so that a re-IP doe…

I had this exact issue in 2008 in SF on Comcast Business. The lease would be dropped and renegotiated every few hours, we could notice it because our SSH tunnels would get torn down. What was annoying was that the business plan had a static IP associated with it, presumably managed by their DHCP setup. It took weeks to convince Comcast the issue wasn’t in our building, and moments for them to fix once they “got it”.

What piece of evidence or logs finally convinced Comcast that the issue was on their side?

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#62

MY INTERNET HAS BEEN GOING OUT AT 3:40pm PT TOO! Sorry for the all caps. I just scrolled through my texts and saw the “internet out?” texts are at the same time. Hopefully that means it’s a provider issue behind us. Wonder what we have in common? - South Lake Tahoe, CA - Spectrum Gig internet - Asus Zenwifi mesh

Hmmm my Spectrum Gig in OR doesn’t go out at 3:40p but it does pretty much every day at 1pm. I was similarly excited to see this post...

Spectrum in Huntington Beach, CA; ours does go out for one minute during the day as well though I didn't think to get systematic and note the times.

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#63
post #42
post #36

At 3:39p, unplug your router and leave it unplugged for an hour, then plug it back in. If the outage then moves to 4:40p every day, it's a 24 hour IP address lease expiring.

Are you talking about IP leases from the router to the local network or IP leases from the ISP to the router?

likely router -> ISP - if it was the devices, they could show up as not being connected to the network. also weird that all devices' leases would expire at the same time

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#64
post #59

Earlier quoted context omitted.

Actually it seems that it is not renegotiating everyday: it is expiring and re-requesting. The way DHCP 'should' work is that when a client is part-way through the lease it should try to renew the IP it already has: * https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Pro... * https://tools.ietf.org/html/rfc2131#section-4.4.5 This allows the client to (hopefully) keep extending the IP it has so that a re-IP doe…

I had this exact issue in 2008 in SF on Comcast Business. The lease would be dropped and renegotiated every few hours, we could notice it because our SSH tunnels would get torn down. What was annoying was that the business plan had a static IP associated with it, presumably managed by their DHCP setup. It took weeks to convince Comcast the issue wasn’t in our building, and moments for them to fix once they “got it”.

I saw this daily drop on several small biz Comcast setups. I recall I found it while looking through all the settings on the router, though I do recall having to call Comcast and they set something so it never happened again. It was the Comcast device with built in wifi. Hardwired didn't glitch, just wifi.

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#65
post #37

Have you tried different WiFi channels? I had a similar experience to this years ago while working at my mother's kitchen table for while. The network would keep going dead and I had no idea what was causing it. Eventually, I discovered that it happened every time she used the microwave to heat up a drink or something. Changing to a different WiFi channel fixed it. :)

I came to suggest the microwave too. The 24h IP reset sounds more likely, but don't discount the microwave, they typically use the 2.4 GHz frequency (as do bluetooth and some wifi networks of course). So, if the 24h IP reset isn't it, try switching to a 5 GHz wifi network to see if the same dropout occurs. If that solves it, it could be a nearby microwave oven.

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#68
What channels do you use on wifi? If you you use DFS channels on 5GHz it could be a radar issue that forces your router to switch channels. I have it as well, but it happens at night so I'm not terribly concerned about it (my router sends me notification thats how I know).

Edit: https://en.wikipedia.org/wiki/Dynamic_frequency_selection

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#69
I will point out to all the people surprised at the time lining up that if the issue is a timer then this is likely very common. There are only 1440 minutes in a day, and waking hours are half that brinignrh it very close to the birthday paradox (double). https://en.m.wikipedia.org/wiki/Birthday_problem.

If you consider 4 time zones in the us it's actually more common than birthdays since you'd divide by four again and people would suspect both a company wide tx synced issue and a per tx issue.

With 23 people about 1/3rd of the people commenting at this point we get a chance of 2 people colliding as:

Full day: 16% 12 Hour: 29% Across time zones: 79%

Re: Every day at the same time, my internet dies for 1 minute. How do I investigate?

#70
I'm actually surprised by all of the comments. The community jumped right into solving OP's problem and has largely ignored the OP's original question of "how could I begin investigating this?" Another variation of this question could be "how do I troubleshoot complex network issues?"

After 20 years of network/system/software engineering across a wide variety of network sizes and levels of complexity, I teach people the following method to troubleshoot complex network issues:

1. Write down the symptoms that you're seeing. How do you know something is wrong? What do you see happening?

2. Draw a diagram that includes all of the components and nodes involved. (this is hard)

3. Develop some hypotheses that could be worth testing. (this is hard and highly variable based on knowledge/experience)

4. Establish a "test plan" that allows you to prove/disprove the hypotheses while making minimal changes to the system. Start from one source device and work your way out to the farthest component you identified in your diagram. Start at the lowest OSI layer and work your way up.

5. Methodically test components, step by step, along the diagram.

6. As you test, record your results. Add new hypothesis to the list as you gather more information but don't start testing them right away! You can develop a new test plan after you finish the first one that proves/disproves your new hypotheses.

7. Repeat all steps, adding new information, until you find a solution.

I know this seems like a lot of steps when you're just troubleshooting a WiFi issue at home--it may be overkill. However, this framework applies to that scenario or when you're diagnosing any complex network issue. You'll learn more about the systems, protocols, and devices that truly make up the Internet than you could ever imagine. Through repetition and experience, these steps will get easier and some of them can happen quickly and in your head.

To help you get started, here's some info for steps 1, 2, and 3:

1. From your post: "3:40pm every day my wifi loses internet access. My devices remain connected to the network, but all traffic dies. Almost exactly 1 minute later everything is resumed"

2. Start your diagram off as an "equipment string diagram". This will include all physical devices between you and the Internet. Along the way, you may need to modify the equipment string diagram to include "virtual" devices, network segments, various protocols/servers, etc. Your diagram should include at least the following items:

  - Your laptop/desktop.  If it's happening to all devices, then pick one.  Identify your IP and MAC addresses
  - The WiFi access point that device is connected to.  Identify the IP and MAC addresses
  - Any switches, firewalls, modems, etc that connect from your WiFi access point to your Internet Service Provider (ISP) and their IP and MAC addresses.
  - The "next hop" from your modem into the ISP's network.  You can use a cloud to represent the ISP's network that you don't know/understand, but always identify the IP address of the device that your modem first reaches.  If you can find the MAC address (or other OSI Layer 2 address) as well, even better.
That's the equipment string between you and "The Internet". For now, you can ignore the complexity inside the ISP's network and beyond. You might have to add more of that later, but start small.

We know there are other components involved in making the Internet work that introduce complexity and can cause issues along the way. Let's list them on your diagram and identify what servers are used and where they might be located.

  - DHCP (local to your network and also between your modem and ISP)
  - DNS (could be local to your network and often is a third party service either run by your ISP or not)
  - Encryption (VPNs, SSL certificates, network device clock settings, etc)
  - IP routing (what devices do IP routing?  Hint: all devices that operate at Layer 3, using IP, do IP routing--including your workstations)
3. Some hypotheses (some were identified in the comments):

  - Is there a device in the equipment string that is rebooting every day?
  - Is DNS intermittently failing?
  - Is DHCP releasing/renewing your IP address assignment?  This could be from your device -> local DHCP server OR your modem -> ISP DHCP server
  - Is there an upstream connectivity issue with your ISP?
  - Is your WiFi access point losing connectivity?
Post reply on HN