Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How well do cars do in crash tests they're not optimized for?

danluu.com

Post reply on HN
Print view View original

121–130 of 352 posts

Re: How well do cars do in crash tests they're not optimized for?

#121

Post by henryfjordan »

ebg13 wrote: post #95
kube-system wrote: post #90

Earlier quoted context omitted.

The big difference with VW is that they put in a mechanism that detects the test and completely changed the behavior of the car for the test.

So they _intentionally_ set themselves up to lie to regulatory agencies and consumers about real world efficiency. That honestly sounds basically the same to me. In both cases the tests are poor approximations, and in both cases someone could accidentally optimize the test, and in both cases someone did it intentionally to deceive people.

Imagine the water heater company didn't know they were gaming the test. They first design a water heater and it gets a B- on the test. Being overachievers, they work hard and submit a second design that gets an A+. They might not realize that both heaters are basically the same with the only difference being some heating element spacing that works better for the test. Both times they submitted a legit design that was the same they'd provide to consumers. Sure, we know the engineers knew what was happening, but we can see how one might innocently arrive in the same scenario. I think it's safe to say the test is flawed.

The VW test is not like that. There's no way to innocently arrive in the scenario they did. They did not game a bad test, they literally lied to the test administrators. The car ran in "clean mode" only if it was in the test environment. If the car ran like it did on the road, they'd have failed (which is how they were caught, with a mobile testing setup).

One of the points in the article is that regulating for safety based on known testing conditions is going to result in over-fitting for the test. The water heater company is guilty of intentionally over-fitting. VW just straight up lied. I don't think those 2 actions are equal, VW is worse, but I agree that both are dishonest to a degree.

Re: How well do cars do in crash tests they're not optimized for?

#122

Post by nitrogen »

itpwang wrote: post #68
Merrill wrote: post #53

>For a long time, adult NHSTA and IIHS tests used a 1970s 50%-ile male dummy, which is 5'9" and 171lbs. ... >For reference, in 2019, the average weight of a U.S. adult male was 198 lbs and the average weight of a U.S. adult female was 171 lbs. So the average mass of the American male has increased 15% in 50 years! No data on whether he is taller, but I'd bet the BMI has gone up.

Referencing the same line, I remember seeing another article on HN somewhat recently about the downside to taking averages of a human to fit specs since the "average"(mathematical mean of measurements) human never existed in reality. Something about fitting pilots to a cockpit?

These "average" cars have pretty terrible seat backs if you happen to be taller than the average. Outright destructive to the spine.

Re: How well do cars do in crash tests they're not optimized for?

#123

Post by bzbarsky »

sjwright wrote: post #55
octagonal wrote: post #14

Earlier quoted context omitted.

The only safe way to drive an F1 car is with remote control.

No, you could also safely drive an F1 car from the driver’s seat by limiting your speed to 40 km/h.

Maybe. Last I checked, the tires on F1 cars are optimized for traction at actual racing speeds (when they get hot and stickier), which means that at low speeds F1 cars actually have much poorer traction than one might think. Only matters on turns, presumably, and 40km/h might be slow enough that it does not matter, but I'd really want to see some experimental data before deciding on anything that happens so far out of a vehicle's intended-use envelope.

Re: How well do cars do in crash tests they're not optimized for?

#124

Post by cortesoft »

ebg13 wrote: post #95
kube-system wrote: post #90

Earlier quoted context omitted.

The big difference with VW is that they put in a mechanism that detects the test and completely changed the behavior of the car for the test.

So they _intentionally_ set themselves up to lie to regulatory agencies and consumers about real world efficiency. That honestly sounds basically the same to me. In both cases the tests are poor approximations, and in both cases someone could accidentally optimize the test, and in both cases someone did it intentionally to deceive people.

The key legal difference is that VW literally behaved differently if a test was running. If they had simply designed a system that tested better than it actually performed (by optimizing the factors tested for), they would not have gotten in trouble.

If the water heater manufacturer had special heating elements that only ran during the test, it would be equivalent.

Re: How well do cars do in crash tests they're not optimized for?

#125

Post by nitrogen »

spott wrote: post #66
strict9 wrote: post #58

Whenever metrics are instrumented that measure productivity, safety, quality, or efficiency, the metrics and metrics alone are what matter, not the over arching goal. Whether it's car crashes, fuel efficiency, lines of code, completed tickets, OKRs, or anything else, players in the system adjust to optimize output for the evaluated metric. A tale as old as time.

Unfortunately, there isn't really a solution to this problem. Metrics are the only way to measure some things reliably and consistently across users, testers and subjects. The goal is to have performance on the metric be as close as possible to the desired characteristic. This isn't always possible (and gets really hard when people are involved), but there isn't really any other solution.

Hidden metrics help, but it is pretty frustrating as an individual to be evaluated based on hidden criteria.

Re: How well do cars do in crash tests they're not optimized for?

#126

Post by gav »

kube-system wrote: post #59
nradov wrote: post #48

Earlier quoted context omitted.

The same discrepancy appears in the 2014 data as well so it's probably not a statistical fluke. Maybe there's a real design flaw in the Juke 4WD model?

The person is not suggesting it's a statistical flaw -- but that it's a self-selection bias. Hypothesis: The people who opt to spend more for a 4wd version are people who know they will use it because they operate the vehicle in inclement conditions, and/or they are uncomfortable with their ability to operate a 2wd vehicle in those conditions and/or they do not have the option to stay home when the weather is bad.

Living in Vermont for five years taught me that while a 4wd is a better choice for bad conditions, it's because they are less likely to get stuck with spinning wheels, but they don't stop any faster than a 2wd.

I would see way more SUVs in ditches than sedans. People tend to overestimate what their 4wd is going to do for them and aren't sufficiently cautious. The first snow/ice of the year was particularly bad.

Re: How well do cars do in crash tests they're not optimized for?

#127

Post by SkyBelow »

ebg13 wrote: post #95
kube-system wrote: post #90

Earlier quoted context omitted.

The big difference with VW is that they put in a mechanism that detects the test and completely changed the behavior of the car for the test.

So they _intentionally_ set themselves up to lie to regulatory agencies and consumers about real world efficiency. That honestly sounds basically the same to me. In both cases the tests are poor approximations, and in both cases someone could accidentally optimize the test, and in both cases someone did it intentionally to deceive people.

There is a difference between memorizing enough information to ace a test and sneaking in notes that aren't allowed to ace a test. And people would also say it is wrong to steal a copy of a test and then memorize the answers to it to ace a test. But what if a professor uses the same test every year (maybe changing a few numbers but in a way that only impacts the calculations, not the way to solve it) and people study just the information needed to answer the test. Is that cheating?

Re: How well do cars do in crash tests they're not optimized for?

#128

Post by [deleted] »

ivanbakel wrote: post #93
henryfjordan wrote: post #85

Earlier quoted context omitted.

VW actively detected if the car was on a treadmill testing setup and electronically changed the engine parameters to make it run cleaner. The water heater was designed with the test in mind but functions the same regardless. VW was clearly cheating whereas the water heater is taking advantage of known deficiencies in the test (is that cheating?).

What was more heinous about the VW case IIRC is that the cleanliness gains the cars made on the treadmill setup were enough to push them into compliance with regulations, which the engines normally didn't meet. "Cheating" a better efficiency rating is significantly less reprehensible than cheating a legal obligation which was legislated for public health and environmental reasons.

[deleted]

Re: How well do cars do in crash tests they're not optimized for?

#129

Post by im3w1l »

cybrjoe wrote: post #82
wscott wrote: post #30

The article reminds me of a story from a friend that was an engineer at a major water heater company. Modern electric water heaters are already fairly efficient, but every model gets tested by the government to be given an efficiency rating. The test consists of replacing the anode with a rod that has a series of temperature probes to measure the water at different levels in the water heater. Since water would form l…

Serious question, how is this different from what VW did? There was a lot of talk about complicit engineers vs. shady management in that discussion. Did this ever make it to production?

Many important differences.

1. Actively detecting test and behaving differently. It's like stealing a test vs teaching to the test.

2. Lower stakes. Health issues are much more serious than inefficiency.

3. It affects the buyer. It's more acceptable for the buyer to be cheated than everyone around them.

4. People could have created these layers by accident. Favouring those who got lucky is unfair.

Honestly I think basically all my gadgets exaggerate how energy efficient they are, by tuning parameters for tests that don't correspond to the real world. My dishwasher has an energy efficient mode, the manual literally says it's just for compliance and recommends other modes. It's just a fact of life.

Re: How well do cars do in crash tests they're not optimized for?

#130

Post by ghaff »

jhardy54 wrote: post #111
shoes_for_thee wrote: post #43

Earlier quoted context omitted.

This might have some useful info for you... https://www.cyclehelmets.org/1026.html death rates per million hours of activity

Do you know what 'residential fire' means? I'm imagining a scenario where my residence is on fire, which would seem to be high-risk (but is somehow the lowest risk).

It's a time-based rate based on (presumably) lifetime hours. Obviously if a residence is on fire, you're at high risk. But the rate of that happening is quite low.
Post reply on HN