All of this testing can be seen as very poor, however, I would argue that we are running the wrong tests. Let me explain...
Imagine you were developing a car. As part of the project you would need to test, test, test and test the drivetrain and the engine. For your test data to be relevant you would want a controlled environment, e.g. car on perfectly flat road with no pot holes, turns or aerodynamic consideration. You could then adjust the variables and check the results - power, torque, emissions, noise level, oil use, fuel use, temperatures and so on.
In this development mode the test rig works pretty good. Figures can be produced that are important for the engine/drivetrain development. Variables can be changed and results measured. The figures - defeat device aside - are actually true with the normal margins for statistical measurement (not every engine will be 100% exactly the same even if it came off the same production batch).
The problem - defeat device aside - is when regulatory bodies use this data for what they need to know, i.e. real world performance. They accept data that they know is not real world and accept it as 'fact'.
If we look at our own testing for building apps, websites and such like we take a modular approach, testing our dev boxes with some type of 'seige' that does not factor in real world internet connectivity and bandwidth, not to mention how customers might use our products in the wild. We can even prove to our clients that their site isn't slow, sharing our metrics with them. Yet, in the real world things don't quite attain those metrics. Luckily nobody gets harmed if our 'squirrel picture app for cats' app falls short.
So, it is the procedures that are wrong and our standards bodies that are doing it wrong (by blindly accepting manufacturers' test data). We are lucky that VW have been cheating as we now are having the conversation about the testing methodology.