Live data from Hacker News

IBM Cloud was down, as well as their status page

cloud.ibm.com

81–90 of 201 posts

Re: IBM Cloud was down, as well as their status page

#81
post #46

Earlier quoted context omitted.

After signing the appropriate NDA's and being vetted by IBM legal of course

Everything I'd say would be public knowledge, of course. I don't work for IBM, I'm a user :).

You never know which way levity will go on Hacker news :-)

Re: IBM Cloud was down, as well as their status page

#83

Honest slightly cynical question: most probably someone inside the responsible team said some day that it would be very stupid to host the status page inside the same infrastructure being monitored, but they were probably ignored... what should that person do now? Say "toldya!" out loud in the postmortem meeting or simply shut up and move on because reality is that we are hired to do some stupid task and not to think…

Never humiliate a coworker in public. Instead say "both options were considered but ultimately it was decided to select option B for reason Y."

So then your recommendation, really, is to just jump ship, and turn your back on useless pieces of shit, vindictively isolating them, and never looking back.

Don't stand up and fight for what's right. Just turn tail and run, under any circumstances that put you in conflict with leadership and their footguns.

Honey, flies, et c...

Re: IBM Cloud was down, as well as their status page

#84

So what are HNers using IBM Cloud for and where do you see that it has an edge over AWS offerings (where an overlap exists, obviously)? (I figure either you’re in devops and you are putting out fires too busy to read this thread or you’re not and your work is halted because of the incident so you might have time to read and reply ;)

If we speak specifically about IBM Cloud vs AWS, we use the Natural Language Understanding API in IBM Cloud and as far as I know the equivalent AWS offering, Comprehend, doesn't provide named entity disambiguation nor links to knowledge graphs (IBM links to DBpedia).

MS and Google do provide those features though.

Re: IBM Cloud was down, as well as their status page

#86
post #79
post #70

Earlier quoted context omitted.

Totally unrelated Jeremy, but did you know the SSL cert on https://minops.com/ has expired? (Was interested to see what you were up to these days, which is how I stumbled on it).

Lol yes. It's intentional. It stops spam bots from trying to sign up, since we aren't open for signups yet. But don't worry, you're not the first to mention it. I suppose I should just fix it and deal with the spam like normal. I liked the unintended effect of cutting down on spam. I guess a lot of spam bots are written on top of standard libraries that reject bad certs. :) Also, this was ironically a great way to pu…

This has to be the most unconventional anti-spam technique I've ever heard about.

Re: IBM Cloud was down, as well as their status page

#87

So what are HNers using IBM Cloud for and where do you see that it has an edge over AWS offerings (where an overlap exists, obviously)? (I figure either you’re in devops and you are putting out fires too busy to read this thread or you’re not and your work is halted because of the incident so you might have time to read and reply ;)

We used Softlayer (rebranded to IBM Cloud, and affected by this) at my last job. For the most part, their service pretty much just works; clearly not today. :)

We had a couple thousand bare metal servers, and barely used any of their API stuff.

As with any facility, there were occasional issues with electrical transfer switches, core router failures, fiber cuts, etc. Stuff happens, but we got pretty good communication, and things got resolved in a reasonable amount of time. Service got noticeably worse after IBM, but we were already planning to move to our acquirers hosting, because that's what happens when you're acquired. Oh, and their load balancers had garbage uptime.

Bandwidth prices used to be pretty reasonable, but they've adopted AWS style obscene pricing. At least they still let you use the private network for free (including to other datacenters).

Re: IBM Cloud was down, as well as their status page

#88
post #79

Earlier quoted context omitted.

Lol yes. It's intentional. It stops spam bots from trying to sign up, since we aren't open for signups yet. But don't worry, you're not the first to mention it. I suppose I should just fix it and deal with the spam like normal. I liked the unintended effect of cutting down on spam. I guess a lot of spam bots are written on top of standard libraries that reject bad certs. :) Also, this was ironically a great way to pu…

This has to be the most unconventional anti-spam technique I've ever heard about.

I stumbled on it by accident. I was lazy and let the cert lapse, but then noticed that spam signups basically stopped. One day maybe I'll make a post about it with graphs, although I'm not sure I actually have the data.

Re: IBM Cloud was down, as well as their status page

#89
post #33

Honest slightly cynical question: most probably someone inside the responsible team said some day that it would be very stupid to host the status page inside the same infrastructure being monitored, but they were probably ignored... what should that person do now? Say "toldya!" out loud in the postmortem meeting or simply shut up and move on because reality is that we are hired to do some stupid task and not to think…

Don’t bite or embarrass the hand that provides you make-work... Seriously they probably tested it and it worked in theory, just not in practice and now they fix it for reals.

These are the same people who recently published a white paper on how they guarantee zero downtime: https://cloud.ibm.com/docs/overview?topic=overview-zero-down...

The idea that they could even get to this point probably seemed unfathomable. It does to me.

Re: IBM Cloud was down, as well as their status page

#90
post #14

Earlier quoted context omitted.

Maybe. About 3/4 of all outages get a post mortem. There's 1/4 of the time they refuse to tell us anything.

Do you actually have data on that or are you conjecturing? Because I would really love to see data about that if it exists somewhere.

I'm talking from experience. Most things do get post mortems, but there's a lot of crap they also don't give us post mortems for "because customer data." It's my number 1 complaint, and I fight with managers about this all the time. We have a ton of hypervisor problems, and a lot of networking issues (generally over private network) and they tend to get very very secretive about it.
Post reply on HN