31–40 of 84 posts
I will always remember this incident as the time when the UK general public were exposed, en masse, to Spring error messages. The confusion caused by ordering a member of the general public not to request a bean from a bean factory in a destroy method implementation still makes me laugh, even now.
This is a shameless indulgence in incompetence and recklessness. They didn't even bother to test large swaths of the transitional data or have a fallback plan if things went wrong.
Most likely their customers will now simply leave and the bank will be shut down.
Earlier quoted context omitted.
True as far as it goes, but "everything broke" as a predictable result of poor decisions they made, such as moving everyone over in one go.
> such as moving everyone over in one go. It's the core banking system of a big bank. Handling consistent state between the old and new systems while progressively migrating customers would probably have been extremely complex. They also get to have a maintenance window (few people will complain if they get warned their bank and all transactions won't work for 2 hours in the middle of the night on Monday). A "big ban…
That sounds like ‘A “big bang” migration makes more sense, if it works.’
Your rollback plan should never be an afterthought. Your rollback plan should be designed like you expect to use it. If your rollback plan is “Burn the ships” and fix-forward, you shouldn’t be working at a bank.
I have a bank account with TSB and got compensation as a result of this mix-up. Some rather personal experiences of the fiasco: – Rather pointlessly, the website changed from being mostly static to entirely written in a very JS-heavy, "dynamic" way. I still can't use it in my normal browser (FF) with its extensions because it relies heavily upon CORS requests and referrer information that my somewhat privacy-paranoid…
I sneer at the emphasis on “1.4 billion records!” in the article as if it’s a lot. At a recent place of employment I created and was responsible for a database that had about that many records and in actuality was a single 2tb postgres db and completely unremarkable. I never claimed to have worked with big data.
Just wondering if the migration disaster at this scale can be avoided using modern cluster and orchestration technology like Kubernetes?
As a sibling comment states a screw up of this magnitude is never simply a technology issue — it requires bad management at many levels.
https://news.ycombinator.com/item?id=16910947
Others
https://hn.algolia.com/?dateEnd=1600300800&dateRange=custom&...
I have a bank account with TSB and got compensation as a result of this mix-up. Some rather personal experiences of the fiasco: – Rather pointlessly, the website changed from being mostly static to entirely written in a very JS-heavy, "dynamic" way. I still can't use it in my normal browser (FF) with its extensions because it relies heavily upon CORS requests and referrer information that my somewhat privacy-paranoid…
Honest question - why do you still have an account with them?
Abusive relationships can trap you, personal or business.
I sneer at the emphasis on “1.4 billion records!” in the article as if it’s a lot. At a recent place of employment I created and was responsible for a database that had about that many records and in actuality was a single 2tb postgres db and completely unremarkable. I never claimed to have worked with big data.
Just wondering if the migration disaster at this scale can be avoided using modern cluster and orchestration technology like Kubernetes?
Still need to write all the procedures, test it, then do it on live system again.
It might make prototyping easier (...or harder) but that's about it
The 250+ page analysis of the incident was an excellent insight into how large IT projects fail: https://www.tsb.co.uk/news-releases/slaughter-and-may/slaugh... money quote: > This situation has all the hallmarks of business management strong-arming the IT organization into an unrealistic timeline. When business leaders push for overly-aggressive timelines, or regulators ask for multiple competing risk frameworks and…
Sadly far too often managers and decisionmakers think timeline and deadlines is a topic for haggle, not discussion. They treat it same as haggling on a bazaar.
If I'm asked for an estimate to do X and I say I can deliver in three weeks, and my boss says customer needs it for golive in three days, I'll try to find some way of making that work. Perhaps they can live without a full solution for the first few weeks, instead requiring only a subset of the requirements in that period. Or, if I insist it just cannot be done, I'll tell the boss who'll try to push back golive if it's important enough.
There needs to be respect for each other and the project though.