31–40 of 137 posts
Aren't there are exceptions? For example, a measure of health might be life expectancy. If this becomes known, is it no longer a good measure? I think it's still a good measure. I mean, how can you get to a life expectancy of 100 without a healthy population?
Earlier quoted context omitted.
Probably more of an incentive for NCO's and Officers to overclaim the results, so they look good to their superiors.
That seems to happen in all wars. Aircraft downed and tanks destroyed in WWII, for example. There were occasions in the Vietnam War where regions were declared to be free fire zones, and everyone within was to be killed, down to the last child and granny.
The key for a good society is regulating the places where making max profits can induce bad behavior and leaving alone where it does not. A hard thing to get right but forcing people to inform you of the ingredients of food for sale, is an example of a good regulation I think.
Aren't there are exceptions? For example, a measure of health might be life expectancy. If this becomes known, is it no longer a good measure? I think it's still a good measure. I mean, how can you get to a life expectancy of 100 without a healthy population?
1) Exclude deaths that'd hurt your life expectancy. For instance, by under counting infant mortality. E.g. does a child who dies during birth count against your life expectancy? Arguments can go both ways, but if your hospital's funding depends on having good life expectancy numbers in your community... well, there you go. IIRC this is one factor that helps explain Cuba's surprisingly high life expectancy numbers, though it doesn't account for most of it.
2) Overtreat end of life issues. Instead of focusing on quality of life and general health throughout the lifespan, put all your funding into heroic interventions that let 90 year olds live on artificial respiration for months. This is what the United States does.
3) Systematically deceive. Have a significant homeless population? Rural population that's deeply unhealthy? Make sure you put a ton of resources into measuring outcomes for your urban population with relatively good health access, and the peasants you miss are just a sign of not getting enough grant money from the WHO. Governments that put resources into getting measurements of the most marginal populations are penalized, while those that deprioritize studies of those marginal populations would have better numbers for their marketing brochures. Even though we'd expect the former to have actual better outcomes than the latter!
"You get what you measure" is the bane of KPIs everywhere. Measure deals closed, and you get deals with no regard for profitability. Measure profitability, and you will burn through your customer base by jacking prices and cutting quality. Etc etc.
When I interviewed for a data analyst position at a known startup, I was asked to define a target KPI for a controlled pseudo-A/B experiment, and I suggested total revenue. The interviewer replied "that's a dumb answer because revenue can be gamed; you should have said # of sales." I was too taken aback to follow up asking why # of sales can't be gamed too.
Any chance the interviewer was overly exit eager ?
Aren't there are exceptions? For example, a measure of health might be life expectancy. If this becomes known, is it no longer a good measure? I think it's still a good measure. I mean, how can you get to a life expectancy of 100 without a healthy population?
Better to measure quality-adjusted life-years, in a way that accounts for the fact that it's sometimes better to take a risk of death with a possibility of full recovery over a guarantee of lost function.
I can't possibly enumerate all possible metrics and their effects, but I bet we could do pretty well at imagining faults for anything that can be constructed.
I wish more people knew and understood this concept. It's important to know the impact that you have on a system, especially at large scale. Decades after Goodhart made this law, we still have shoddy credit risk measures that put tens of trillions of dollars at risk just in the US alone. Hell, the last recession alone was mostly caused by the government's choice of credit worthiness metrics.
Can you explain how the recession was caused by "government's choice of credit worthiness metrics", instead of rollbacks of regulation?
The government's choice to allow securities rated by S&P/Moody's to be treated as gospel allowed/encouraged banks, insurance companies, GSE's and pension funds to take far more risk than was systemically reasonable.
Earlier quoted context omitted.
When I interviewed for a data analyst position at a known startup, I was asked to define a target KPI for a controlled pseudo-A/B experiment, and I suggested total revenue. The interviewer replied "that's a dumb answer because revenue can be gamed; you should have said # of sales." I was too taken aback to follow up asking why # of sales can't be gamed too.
Number of sales (or customers) is a vanity metric favored by startups looking to entice investment. Revenue is a metric useful for measuring actual business health. Any chance the interviewer was overly exit eager ?
The breaking of this law when one uses profit as the measure of a successful company or business is why I think that a well set up market system actually works. People and companies can set up making a profit as their target and it is still a good measure of success, because it is actually what you want. The key for a good society is regulating the places where making max profits can induce bad behavior and leaving a…