Skills Don’t Pay the Bills

I was given a good piece of advice when I started work: "A process engineer's job is to make themselves obsolete." I've striven to make sure that I did everything I could to make sure I wasn't needed anymore.

And I'm happy to say that the processes will keep me employed for as long as I'm willing. Not from any self-serving malice, but rather that in the end it's terribly messy. The sheer inelegance of industrial manufacturing seems to defy strong automation at the meta-level. And not just because the machines are all only halfway obsolete, new, deprecated, upgraded, documented, up-kept, cleaned, and run (halfway to spec).

It's because to a computer, this mess quickly degrades into special cases. Lots of them. There actually are some shockingly clever software packages out there capable of the early stages of what you're describing, and - this is the kicker - they all require specialized engineering support to create, install, train, and use. The end product can be handled by a floor worker or management, but at some point you've got to explain the situation to a computer somehow, and that ends up being as complicated as programming.

For a much better written punchline, I'll defer to the short story Profession by Asimov [1]. I think it's cynical to think we'll never be able to solve the 'original thought' problem, but I also think it's easy to underestimate it. Things really are different today than 30 years ago, so perhaps a few more iterations will lead to tape-machines that can program the tape-machine programmer tape-machines. [2]

[1] http://www.abelard.org/asimov.php

[2] Perhaps the worst part is we sorta already have this. There was just a post on the LLVM [3], which discussed how a computer translated and optimized a division command into machine code. That used to take a lot of work, and it took Real Programmers to optimize that code before this sort of automation. So it's not inconceivable that your ideal comes true. But I am cynical enough to think sami36 is right [4]: Doodleware is past the horizon. But computers that can talk to computers is probably in sight.

[3] http://eli.thegreenplace.net/2012/11/24/life-of-an-instructi...

[4] http://news.ycombinator.com/item?id=4827459

"Data science" means many things to many people. Some shops want BI folks who can code. Others want people who can keep up in deep learning threads (like the one on HN's front page) and know about Hinton's work. Others still want HDFS badasses who can distinguish between Cascading and Cascalog (lol).

Shops classify all of these things as "data science" when they're really very different (respectively: Excel/scripting BI analyst; ML researcher with a focus on deep learning and neural networks; data infrastructure developer with broad exposure to MR frameworks)

This is stupid. I worried about checking off these boxes until I realized recruiters are doing the 1990s Java thing all over again. This time, though, the buzzwords are Hadoop, information retrieval, and Andrew Ng. I think it's best to ignore this, keep working hard, and grow your employment options.

Now, assuming blue collar is done, as robots can take care of most of the tasks, could someone explain why we need workers to oversee the computers?

The way I see it, you need a really expensive robot, and you need a really expensive programmer to make it work; everything in the middle is cheap. Software should be able to distill anything that is happening on the robotic side and present it in such a way that a trained mechanic would understand and be able to fix it. If that's not the case, then we need better developers (and in this case, perhaps, better designers and human-factors engineers) to write better software.

Now, a few folks mentioned that what you need is really a mixture of experience in the mechanical side of things and understanding of the software - presumable that allows you to react and solve problems whenever something goes astray. Well, that could probably be solved by having a few engineers on staff who would help when needed.

That leaves us with yet again, fairly simple mechanical labor. Perhaps then, the article is right, we have a serious education problem. Those who attain enough education, leap forward and presumable learn more to subject themselves to mundane mechanical tasks, while those who would be greatly fitted to do the jobs, actually don't have enough education to understand even the most basics.

Someone's suggested comparing the math needed at Mc'Donalds with the math needed at one of these factories. I'd be curious to know too; although I suspect that in McDs all the calculations are done by a computer and all humans need to do, is simply not to f-up. Even then, when humans fail to add 2+2, all they lose is an occasional McFlurry, while at a factory they could impact tens of thousands of dollars at once.

So, here we have a conundrum. We need labor to work the $10 jobs, but the pool of employees is simply atrocious. At the same time, qualified labor has better things to do with their time. Now, back to my questions - what's the solution?

On the other hand, I have never ever found the requirements for a role to be nothing but mostly bullshit. So they don't really help you much. Your best bet if you are not going through a referral is to attempt to get to a phone screen where the Engineer/Data Scientist will be kind enough to spare some time at the end of the interview to tell you what on earth they are actually looking for. You could get lucky and learn just enough to terminate the interview process at this time. The worst situation is where you go there and realize that either a) You are not what they are looking for which is a half a day wasted or even worse b) They have not completely realized what they are looking for.

Me? I have made a decision that I will either apply for software engineering positions where I will maybe be one of the Machine Learning folks or apply to data science positions only if I know some one in the team who can explain to me what on earth that company is actually looking for.