Yeah, it's very opaque, way more than most consumers know.
For example you don't have a single FICO or XPN score. When a lender pulls a report they pick an option for the purpose, and that picks one of a couple dozen different models. All the models come from Fair Isaac but are tweaked to context.
I have a bit of a unique perspective on all this because my job was basically to research negative items on mortgage applicant credit reports, and if I could get the creditor to say something that met one of a couple dozen criteria then I could pull the item off the report and resubmit it to the big 3 for a new score.
A big part of my job was explaining to loan officers what the score impact of particular changes to the report would be. Fair Isaac obviously keeps the details of their models proprietary, but doing my job you'd accumulate an intuition for what would do what. This was all in the early 00's and I've no doubt things have changed in detail but not really in the overall picture I'm presenting.
The main thing to understand about credit scores is creditors want to see you using credit but paying reliably. They want to see you carry some balance because that's where they make money, but they don't want to see you running into what looks like unsustainable balance growth relative to your payment history. Retail credit cards are a positive signal because it shows you're a good little consumer that will float a balance for a few months to buy that new whatever but you always end up paying on time.
The mistake a lot of people think about these scores is that they're some sort of measure of personal fiscal discipline. They aren't. They're a score of how likely you are to make a lender money as a borrower. They want debt addicts that pay interest reliably, something that is pointed a different direction from personal fiscal prudence in most cases.
As long as I'm rambling a couple other tips:
Creditors will often remove negative payment history if you simply ask. You've got nothing to lose by trying, and it works doubly well if you're applying for a new product at the same bank. There is no more effective bank customer service agent than a loan officer determined to get that commission. They will go on a hilarious scorched earth warpath of conference calls with the borrower to get it done.
Most lenders are only really interested in the last 2 years of history. If you're in a bad spot just make getting 2 years of clean payments on a couple sources of credit your goal. Get a secured card if you have to. Use it for ordinary daily expenses vs a cash or debit card, and pay it off each month.
If you want to game the system, the most effective way is to file a dispute with the big 3 on Monday, then apply for the loan or whatever on Tuesday. Disputes temporarily knock items off the report, so you can try to work within the lag time of the bureau processing the dispute. This is particularly effective vs Transunion, which is the main reporter for collection agencies and other really bad stuff on reports, and also is a hilariously lazy and incompetent company.
As to your last point I totally agree about that part. But it's getting hard to avoid the shady in banking. I worked for Wells Fargo, and it was very apparent to me that quality control was structured to whitewash not find fraud. So much obvious fraud that met the law and rules came across my desk. I was totally unsurprised by the 2008 crash as well as the story about Wells Fargo branch managers opening second accounts for customers with forged permission.
The whole credit reporting industry needs severe reform, but it's not even a visible issue on capital hill.