Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What the Fastly outage can teach us about writing error messages

onlineornot.com

Post reply on HN
Print view View original

11–20 of 137 posts

Re: What the Fastly outage can teach us about writing error messages

#11

Post by bombcar »

SilverRed wrote: post #6
xwdv wrote: post #4

Good error messages are hard. There’s so many things that can go wrong, and each needs its own custom explanation. Never really found a good way to organize this. And since errors should be uncommon, it feels like a waste of time coming up with thoughtful messages.

The problem is that you have to know exactly what has gone wrong to show a proper message. But if you know what is wrong, most of the time the correct action is to fix the program so it does not go wrong instead of showing a better error. With the main exception of user submitted data validations where it is up to the user to submit correct data. For the average web service, there the error is a stack trace and there…

Even if there’s usually nothing the user can do I still like to have all the stack traces, etc visible - because sometimes there’s enough of a hint that you can get a workaround.

A number of DNS failures I’ve worked around with the hosts file.

Re: What the Fastly outage can teach us about writing error messages

#12

Post by marcosdumay »

hvocode wrote: post #3

I've had more than one intern or young engineer that I've had to teach to resist the urge to put witty comments and jokes in project docs, comments, or errors. Especially in error messages or things that can be exposed to users/customers. Even if it's an error you think only the dev team will see, you never know if it will make its way out to audiences you didn't intend. Somehow I doubt people faced with this error m…

I'd say that some safe joke on the end of your error message is perfectly fine, as long as you are perfectly clear where the problem is. Just be very sure not to unintentionally blame the user. It is fine even if the user won't get the joke.

Software is a highly personal and creative thing, it should have a personality. The everything gray enterprise spaces have gone too far already. Also, I really miss Linux yelling on panics.

Now, the same joke on a highly visible position or on the beginning of the message is harmful and will impede people from solving the problem.

Re: What the Fastly outage can teach us about writing error messages

#13

Post by res0nat0r »

gonzo41 wrote: post #5

So that looks like a stock Varnish 503. I'm pretty sure the magic behind fastly is a ton of Varnish cache. What's interesting (VCL can be unforgiving), also that it was a big central stuff up. Which makes me think they have layers and layers of caching in depth and some top level config just blacked out everything. I kind of would have thought they'd have lots of small instances.

Fastly is indeed powered by a lot of Varnish magic. My company used(s) them for the CDN for some of our large website frontends, and they have engineers which will help generate the insanely convoluted and large VCL templates which can be setup in your account and will handle all of the magic routing / caching / much more complex things you need for your origin.

Re: What the Fastly outage can teach us about writing error messages

#14

Post by res0nat0r »

orf wrote: post #8
kajaktum wrote: post #7

But who's going to host that nice unicorn images? and those nice CSS? I agree with putting up a link tho.

Embed both in the page.

I suspect most folks, especially ones which operate at GB/s rates serving traffic would not like to serve up a 100kb base64 encoded image every time an error is thrown.

Re: What the Fastly outage can teach us about writing error messages

#15

Post by orf »

res0nat0r wrote: post #14
orf wrote: post #8

Earlier quoted context omitted.

Embed both in the page.

I suspect most folks, especially ones which operate at GB/s rates serving traffic would not like to serve up a 100kb base64 encoded image every time an error is thrown.

Sure, which is why they probably wouldn’t serve up a 100kb base64 encoded image every time an error is thrown.

It’s more than possible to embed a small vector image to add some humanity to an error page without breaking the bank, bandwidth-wise

Re: What the Fastly outage can teach us about writing error messages

#16

Post by gonzo41 »

res0nat0r wrote: post #13
gonzo41 wrote: post #5

So that looks like a stock Varnish 503. I'm pretty sure the magic behind fastly is a ton of Varnish cache. What's interesting (VCL can be unforgiving), also that it was a big central stuff up. Which makes me think they have layers and layers of caching in depth and some top level config just blacked out everything. I kind of would have thought they'd have lots of small instances.

Fastly is indeed powered by a lot of Varnish magic. My company used(s) them for the CDN for some of our large website frontends, and they have engineers which will help generate the insanely convoluted and large VCL templates which can be setup in your account and will handle all of the magic routing / caching / much more complex things you need for your origin.

Yeah I've used varnish before. I like it a lot, but I do think coloring inside the lines as much as possible is probably the safest thing. Some of the VMOD's and more exciting things you can do with varnish become really trick to manage.

Re: What the Fastly outage can teach us about writing error messages

#17

Post by function_seven »

aniforprez wrote: post #10

Reminds me of the quintessential tweet about error messages https://twitter.com/cherrikissu/status/972524442600558594?s=...

With a small group of Corporate Memphis people frowning or looking bewildered.

Give me “Unspecified error” any day over that.

Re: What the Fastly outage can teach us about writing error messages

#18

Post by sachinag »

xwdv wrote: post #4

Good error messages are hard. There’s so many things that can go wrong, and each needs its own custom explanation. Never really found a good way to organize this. And since errors should be uncommon, it feels like a waste of time coming up with thoughtful messages.

Good error messages have a template: a UUID, a description, and a suggested solution. This gives folks enough to try to solve it themselves and also enough to Google / grep with.

Re: What the Fastly outage can teach us about writing error messages

#19

Post by donavanm »

I'm a principal at AWS and spent years dealing with this on CloudFront and Route 53. A lot right, and a little off, in the post. Yes, we should definitely communicate to the viewer what's happened and appropriate next steps if any. But there are a handful of challenges that Ive not seen reasonably solved.

The first problem is that different viewers need different context, which is especially true for service providers like CDNs. Telling an end user "the page is taking too long" makes sense. But what if its the CDN customer (developer), theyre going to need request IDs and other diagnostics just like Fastly did display. Playing the "we need the RID from the headers you didnt know about to capture" game is a losing proposition. The post and it'd "good" examples suggest a way to contact support, but there is ~0% chance support can help without additional information that isnt included.

Expanding on that the post is making really dangerous assumptions and carrying those through to the semantics of the text. Exposing authnz failures as Access Denied vs Not Found is problematic in its own right. Going further to tell the user "You need to login first" is asking for pain in the other 10% of cases where it's a credential or authz problem. Again, different users need different context. And there's no way to know what context to provide apriori.

Lastly it's somewhere between incredibly hard to impossible to distinguish the cause ("why") on a per request basis. Especially for service providers like CDNs. The CDN has a cache miss, but can't retrieve the content. Is it a 504 Gateway Timeout because the origin is not responding, 502 Bad Gateway because the origin TLS is broken, 502 because the local clock is off, or a 503 Service Unavailable because of an internal service timeout? Even if you can distinguish what does the CDN represent to the end user; a general fault in the CDN? A specific failure of the origin? Or a simplistic "unavailable." Again, different viewers need different context and semantics. Which you're not going to do at thousands or millions or tps.

Re: What the Fastly outage can teach us about writing error messages

#20

Post by calvinmorrison »

aniforprez wrote: post #10

Reminds me of the quintessential tweet about error messages https://twitter.com/cherrikissu/status/972524442600558594?s=...

Right up there with Slack etcs fake loading messages

"Gearing up the dildonator"

"Implicating the fairies"

"Hogtying George Bush"

Dude - just give me a spinner or a progress bar, and if something errors during the load out give me some sort of stack trace or error ID I can use to help

Post reply on HN