Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The FAA’s Hiring Scandal

tracingwoodgrains.com

Post reply on HN
Print view View original

551–560 of 724 posts

Re: The FAA’s Hiring Scandal

#551

Post by varloid »

PaulDavisThe1st wrote: post #525
varloid wrote: post #508

Earlier quoted context omitted.

> Performance on the AT-SAT is not job performance. No, but it was the best predictor of job performance and academy pass rate there was. https://apps.dtic.mil/sti/pdfs/ADA566825.pdf https://www.faa.gov/sites/faa.gov/files/data_research/resear... (page 41) There are a fixed number of seats at the ATC academy in OKC, so it's critical to get the highest quality applicants possible to ensure that the pass rate is as hig…

That is NOT what the first study you've cited says at all: > "The empirically-keyed, response-option scored biodata scale demonstrated incremental validity over the computerized aptitude test battery in predicting scores representing the core technical skills of en route controllers." I.e the aptitude test battery is WORSE than the biodata scale. The second citation you offered merely notes that the AT-SAT battery is…

> I.e the aptitude test battery is WORSE than the biodata scale.

You're mistaken, it's the opposite. The first one found that AT-SAT performance was the best measure, with the biodata providing a small enhancement:

> AT-SAT scores accounted for 27% of variance in the criterion measure (β=0.520, adjusted R2=.271,p> In other words, after taking AT-SAT into account, CBAS accounted for just a bit more of the variance in the criterion measure

Hence, "incremental validity."

> The second citation you offered merely notes that the AT-SAT battery is a better predictor than the older OPM battery, not that is the best.

You're right, and I can't remember which study it was that explicitly said that it was the best measure. I'll post it here if I find it. However, given that each failed applicant costs the FAA hundreds of thousands of dollars, we can safely assume that there was no better measure readily available at the time, or it would have been used instead of the AT-SAT. Currently they use the ATSA instead of the AT-SAT, which is supposed to be a better predictor, and they're planning on replacing the AT-SAT in a year or two; it's an ongoing problem with ongoing research.

> I'd also say at a higher level that both of those papers absolutely reek of non-reproduceability and low N problems that plague social and psychological research. I'm not saying they're wrong. They are just not obviously definitive.

Given the limited number of controllers, this is going to be an issue in any study you find on the topic. You can only pull so many people off the boards to take these tests, so you're never going to have an enormous sample size.

Re: The FAA’s Hiring Scandal

#552

Post by Xelynega »

potato3732842 wrote: post #218

Complaints about controller shortages and 6-day weeks being the norm and whatnot go back into the 00s. Why the hell was anyone doing anything to restrict the hiring and onboarding pipeline in the first place? The alleged motivation barely even matters. Heck considering the attrition rate of the career path it would arguably be acceptable if they juiced their hiring pipeline with their preferred demographics. I've see…

Do you honestly believe that hiring rate is determined by DEI departments and not budgets?

Do you really, honestly believe that the FAA was using these practices to hire less people and not just hire the people they want to hire in the limited positions?

Why would they go to such absurd lengths when they could just say "we can't hire more people because we can't afford it"...

Re: The FAA’s Hiring Scandal

#553

Post by clutchdude »

stackskipton wrote: post #542
clutchdude wrote: post #523

Earlier quoted context omitted.

Here ya go: Academy attrition on page 38. https://www.faa.gov/air_traffic/publications/controller_staf...

Figure, it was in a PDF that search engines had trouble scraping. I feel like FAA is burying this data on purpose because it looks terrible. Reading deeper, on page 40 that has historical data, starting FY14 when this survey had been implemented and initial class hired, Academy Training Attrition appears to be much higher though all I can base this on is comparing bar graph sizes. So yes, this change to hiring proces…

Possibly but I'd argue it's far from a smoking gun.

The sequester of 2013 did a number on things and they hired to maximum capacity in the years after to make up for lost time. It stands to reason that by filling training to the max, they'd have more washouts due to lack of more attention during training.

> The sequestration in 2013 and subsequent hiring freeze resulted in the FAA not hiring any new controllers for nearly 9 months across FY 2013 and FY 2014. The effects of this disruption on the hiring pipeline, as well as the FAA Air Traffic Academy’s operations, were substantial.

https://www.faa.gov/air_traffic/publications/controller_staf...

Re: The FAA’s Hiring Scandal

#554

Post by TraceWoodgrains »

armada651 wrote: post #534
ramblenode wrote: post #262

The story is really worth a read. The writing speaks for itself: > The biographical questionnaire Snow referred to as the “first phase” was an unsupervised questionnaire candidates were expected to take at home. You can take a replica copy here. Questions were chosen and weighted bizarrely, with candidates able to answer “A” to all but one question to get through. Some of the most heavily weighted questions were “The…

Those two questions are cherry-picked to imply the questionnaire was specifically designed to only let people pass who preformed badly academically. However there are several other questions that specifically ask for the applicant's average grades and anything less than an A grade will not give you any points. The problem is that the test is completely arbitrary with no rhyme or reason to it, not that it was designed…

To clarify, I picked those two questions not to imply a focus on bad academic performance but because they are both a) absurd/arbitrary and b) the highest-weighted questions by far.

Re: The FAA’s Hiring Scandal

#555

Post by Aloisius »

tim333 wrote: post #345
apical_dendrite wrote: post #148

I'm fairly certain this was an example of overfitting and Freedman's Paradox, not deliberate cheating. Let's say you have a completely random data set. You generate a bunch of random variables x1 through xn and a random dependent variable y. Then you poke around and see whether any of the x variables look like they might predict y, so you pick those variables and try to build a model on them. What you end up with is…

Have you looked at the info on the test here https://kaisoapbox.com/projects/faa_biographical_assessment/ ? (copied from another post) To pass the test you have to click A on all 62 questions apart from question 16 where you have to click D to say your lowest grade in school was in history. The thing's a complete travesty.

You don't have to do that to pass the test. The max score possible is 179. One can pass the test without answering either of the worst subject questions "correctly."

Also answering answer A to 23 (>20 hours/week paid employment last year of college) would logically conflict with answering A to 56 (Did not attend college).

Re: The FAA’s Hiring Scandal

#556

Post by davorak »

SR2Z wrote: post #168
legitster wrote: post #123

Earlier quoted context omitted.

> and the dominant regime at the time downplayed / ignored the DEI elements because that was supported by their ideology In the eye of the beholder. The current regime is upplaying the DEI elements because of their ideology. The difference though is, unless everyone involved has a time machine, using current cultural agenda items and going back in time and attributing them to people is always going to be wild specula…

> using current cultural agenda items and going back in time and attributing them to people is always going to be wild speculation. I'm as blue as they come, but let's not mince words. This was a racial equity policy. Like a lot of them, it was designed by idiots and/or racists. Much like the elite college admissions lawsuit, we don't need to guess at people's ideology - they WROTE DOWN that the cognitive test "disad…

I agree with calling people out.

> This was a racial equity policy. Like a lot of them, it was designed by idiots and/or racists.

So a policy can be labeled an 'equity policy' and have nothing to do with equity in either intent or result, which is what I would expect from an 'equity policy' written by a racist.

Call it corruption, call it fraudulent activity, but it does it seems like there was only lip service to equity. So why would you call it DEI or equity or anything similar?

Company A: Our equity policy is to only hire white men! We are proud of how we are striving towards equity with our new DEI policy.

observer: Damn those DEI policies ruining everything.

To me it is obvious you do not blame 'DEI policies' but the leadership and corruption in Company A.

Re: The FAA’s Hiring Scandal

#557

Post by [deleted] »

tstrimple wrote: post #547
Aloisius wrote: post #546

Earlier quoted context omitted.

It's the first question that provides points, not the only one.

It clearly states that if you choose anything other than science for that question, you get zero points. It's literally in the quiz and captured in a screen shot. How is this controversial or confusing at all?

[deleted]

Re: The FAA’s Hiring Scandal

#558

Post by thaumasiotes »

amluto wrote: post #548
thaumasiotes wrote: post #466

Earlier quoted context omitted.

There are only so many black people in the country. Every skilled job has this problem; poaching can make you look slightly better but it does nothing overall and will make wherever you poached your qualified black applicants from look worse.

> There are only so many black people in the country. The US population is around 1/8 black. Which means, if every kid has an equal opportunity (in an absolute sense or on average) to develop the requisite skills to be an air traffic controller and if every kid was equally inclined to apply, and the application process were fair, then eventually around 1/8 of air traffic controllers would be black. Which seems like a…

> Which means, if every kid has an equal opportunity (in an absolute sense or on average) to develop the requisite skills to be an air traffic controller and if every kid was equally inclined to apply, and the application process were fair, then eventually around 1/8 of air traffic controllers would be black.

It doesn't mean that at all.

Well, depending on what you mean. It could just be that your premise is known to be false.

Re: The FAA’s Hiring Scandal

#559

Post by bimguy »

K0balt wrote: post #455
Aurornis wrote: post #380

Earlier quoted context omitted.

Neither the FAA situation nor the article are about neurodiversity. > We need to stop treating neurodiversity as if it’s a scale from good to bad. It’s just a kind of diversity. In the situation of hiring people for specific jobs, filtering for a perceived "neurodiversity" would have no scientific basis. Fortunately, hiring doesn't work this way. The idea is to hire for people who are qualified for and capable of the…

I think you maybe misunderstood what I was saying. I’m saying that neurodivergence is why some people thrive at certain jobs that others would find exhausting. Ergo we should test for ability, not some arbitrary representation of race, sex, or other non-task related metric.

Testing for ability is exactly what they were originally doing before the NBCFAE got involved in the FAA's well established hiring process.

I get your point, but to be clear this is already what they were doing since 1989 up until 2013.

Re: The FAA’s Hiring Scandal

#560

Post by stackskipton »

clutchdude wrote: post #553
stackskipton wrote: post #542

Earlier quoted context omitted.

Figure, it was in a PDF that search engines had trouble scraping. I feel like FAA is burying this data on purpose because it looks terrible. Reading deeper, on page 40 that has historical data, starting FY14 when this survey had been implemented and initial class hired, Academy Training Attrition appears to be much higher though all I can base this on is comparing bar graph sizes. So yes, this change to hiring proces…

Possibly but I'd argue it's far from a smoking gun. The sequester of 2013 did a number on things and they hired to maximum capacity in the years after to make up for lost time. It stands to reason that by filling training to the max, they'd have more washouts due to lack of more attention during training. > The sequestration in 2013 and subsequent hiring freeze resulted in the FAA not hiring any new controllers for n…

Yep, hmmm. It just sucks that this data which should be easy to find is not and FAA clearly has since they put out the report with it.
Post reply on HN