> ... but as of 22:28 UTC Facebook appears to be ... Someone assumed London==UTC, when London is 1 hour ahead :) that was actually 21:28 UTC
Timezones are the most annoying thing... right after encoding
Understanding how Facebook disappeared from the internet
21–30 of 280 posts
Re: Understanding how Facebook disappeared from the internet
#22[deleted]
Re: Understanding how Facebook disappeared from the internet
#23> ... but as of 22:28 UTC Facebook appears to be ... Someone assumed London==UTC, when London is 1 hour ahead :) that was actually 21:28 UTC
Timezones are the most annoying thing... right after encoding
> right after encoding
No joke. Today I ended up writing a whole essay explaining the issue I was having and almost sending it off to the core developers because I thought I had discovered an issue with the actual language. The bug was because I had forgot to convert too&from utf-8 in these two procedures:
proc 2Hex { input } { binary encode hex [encoding convertto utf-8 "$input"] }
;# Converts base32 string data to base16
proc 2Base { input } { encoding convertfrom utf-8 [binary decode hex "$input"] }
;# Converts string hex data to base32
Re: Understanding how Facebook disappeared from the internet
#24The thing is, because the deletion of the BGP, they have ruled out the cyberattack but what happened doesn't make sense
Re: Understanding how Facebook disappeared from the internet
#25Earlier quoted context omitted.
Do you really think that Facebook is going to write one right now? It's good that we have some coverage from companies that have some stake in the game because they are also affected by the outage, even if only partially.
Facebook has to make some public statement; the shareholders will demand it. I expect the detail level to be roughly "an automated system pushed a broken configuration"; that is to say, there probably won't be any interesting information at all for the Hacker News crowd. I doubt that this was caused by "hackers" or "hostile governments" or "dissident employees upset about Facebook privacy issues", and also doubt that…
The fact is, it's altogether likely that they could be legally require NOT to make such a statement outlining the cause if it was a hostile actor. I've felt a distinct change recently. The US government is not messing around about cyber security anymore.
The guys with the blue windbreakers show up, I'd pretty much say "yes, sir." Of course, I don't have FB's power, but I don't think it matters.
Re: Understanding how Facebook disappeared from the internet
#26Earlier quoted context omitted.
Timezones are the most annoying thing... right after encoding
> right after encoding No joke. Today I ended up writing a whole essay explaining the issue I was having and almost sending it off to the core developers because I thought I had discovered an issue with the actual language. The bug was because I had forgot to convert too&from utf-8 in these two procedures: proc 2Hex { input } { binary encode hex [encoding convertto utf-8 "$input"] } ;# Converts base32 string data to…
Oh TCL. I didn't miss you.
Re: Understanding how Facebook disappeared from the internet
#27This is a great write-up, but one thing I don't understand is why the effect of withdrawing the BGP prefixes was instantaneous (if I understand that correctly), but it's taking hours (so far) to re-announce the prefixes. Why would it take so long to flip the switch back the other way?
I’m pretty new to BGP, but I’d imagine that cutting off access to an AS is fast because all it takes is for the neighbouring routers update their routes. At which point any traffic that makes it that far is simply dropped.
Whereas to make an announcement, the entire internet (or at least all routers between the AS and the user) need to pickup the new announcement.
(Note: I still need to read the article)
Re: Understanding how Facebook disappeared from the internet
#28> ... but as of 22:28 UTC Facebook appears to be ... Someone assumed London==UTC, when London is 1 hour ahead :) that was actually 21:28 UTC
No matter what time of year it is, people tend to use 'EST' for 'Eastern Time' even when we might be in Eastern Daylight Time rather than Standard.
It's especially annoying when dealing with multiple countries that may or may not be using Daylight Saving Time.
Re: Understanding how Facebook disappeared from the internet
#29This is a great write-up, but one thing I don't understand is why the effect of withdrawing the BGP prefixes was instantaneous (if I understand that correctly), but it's taking hours (so far) to re-announce the prefixes. Why would it take so long to flip the switch back the other way?
I haven't been following closely, but I think once they moved the prefixes they could no longer access the routers. Coupled with barebones staff at the data center due to the pandemic, and all internal communication being disrupted. Though I really expected it to be up within an hour or two.
Yeah, I think that is true. If you look at the Update near the end of the Cloudflare article there is a huge spike in the BGP activity (I assume re-announcing all of the routes). So that part of it was relatively instantaneous after they got all of their ducks in a row actually getting to the routers and locating the BGP from some earlier version before it went offline this morning that they could use.
Re: Understanding how Facebook disappeared from the internet
#30> ... but as of 22:28 UTC Facebook appears to be ... Someone assumed London==UTC, when London is 1 hour ahead :) that was actually 21:28 UTC
Timezones are the most annoying thing... right after encoding
I personally find timezones more annoying. At least with encoding once you figure things out it will work indefinitely. Timezones can simply change from under you with or without notice.