The position's still open. It's ironic that it requires "Stay up-to-date on new AI technologies, including LLMs and generative models. Prototype and test new technologies to evaluate quality and improve performance."
Interviewing a software engineer who prepared with AI
171–180 of 731 posts
Re: Interviewing a software engineer who prepared with AI
#172Potentially important side points, since not everyone knows, and we don't want anyone to learn a mistake by example: 1. Don't use blur to redact documents. Whatever blur was used can probably be reversed. 2. Don't try to hide the identity of someone you're talking about by redacting a few details on their resume. With the prevalence of public and private resume databases, that's probably easy to match up with a name.
> Don't use blur to redact documents. Whatever blur was used can probably be reversed. I just got mosquito noise when I sharpened. Are you confusing blurring with pixelating? As long as the blur is strong enough, there's no way to get the text back.
Regular sharpening doesn't work.
But a deconvolution filter will. You can't do it in Photoshop but you can with a dedicated tool that tries different deconvolution kernels until it finds one that matches the exact original blur function.
This is how you can remove motion blur from a photo due to camera movement, for example. It's wild how much information is still there, in the exact precise levels and shape of the blur.
There are limits of course, but they're much further than you might expect.
Re: Interviewing a software engineer who prepared with AI
#173Earlier quoted context omitted.
Excellent points. Building on that, if the people who are bothered by that leave or withdraw, won't the workplace come to be dominated by people who aren't bothered by that? If so, a question is why they aren't bothered by that. Is the culture then cold-hearted? Mercenary? Sociopathic? Oblivious?
People who stayed long enough adjusted, but it didn't mean they were cold-hearted. They just realized that there was more to the story that they saw. The real challenge was when recent hires would see it and get spooked. One person would get fired and then two people around them would panic and start looking for other jobs. Several people panicked and jumped right back into their previous jobs. It was also tough when…
I feel like we're working at the same company. Not just this comment but your others on the same topic. I've seen all the exact same mistakes over the last year. The company wants to grow fast so hires quickly, but then the people hired quickly underperform, so then they're fired quickly, but firing people quickly results in fear, grief and guilt for everyone who hasn't been fired "this time". The top talent never feel comfortable in this cold mercenary culture, so they don't settle in and soon move onto somewhere less cut-throat.
Re: Interviewing a software engineer who prepared with AI
#174An interesting story! I've also had an AI cheater during phone screen, but they were pretty clumsy... A question of form "You mentioned you used TechX on your resume, tell me more what you did with it" was answered with a long-winded but generic description of TechX and zero information about their project or personal contribution. Another thing that I can take away from that is "take home project" is no longer a goo…
> Another thing that I can take away from that is "take home project" is no longer a good idea in AI times Take-home projects were never meant to be evaluated in isolation. It was common for candidates to have their friends review the take-home or even do it for them. You had to structure the take-home so the candidate could then explain their choices to you and walk you through their thought process. When you got a…
The difference is that AI can now feed them explanations as well. Their friends (who IME were usually also mediocre coders: everyone I've seen who actually did well on a take-home actually was that good) didn't have the patience to sit around and help them memorize a bunch of extra nonsense.
At some point it feels like it would be easier to just get good at programming, and yet...
Re: Interviewing a software engineer who prepared with AI
#175Earlier quoted context omitted.
I've had situations where I submitted a take-home exercise, only for me to get feedback that it didn't match their required level. After some back & forth I was able to (politely) prove their feedback was not correct, which actually granted me a follow-up interview. Unfortunately, this was a unicorn, most companies don't give feedback, let alone admit they were wrong. But, take-home is preferred, I want to use my IDE…
>When an HVAC company hires a new tech HVAC has certifications you can get. We should strongly consider this in our industry. I don't think its an unreasonable compromise, especially now with the advent of LLMs.
We have some certs. The problem is that software development is about thirty different skills in a trench coat, and half of them we don't know how to evaluate (like slicing, or abstraction.)
What ends up happening is that our certs end up being a bunch of multiple-choice questions that check people's ability to memorize trivia.
It is more like having a Certified Novel Writer or Certified Mural Painter or Certified Graphic Designer certificate than it is like HVAC or welding.
Re: Interviewing a software engineer who prepared with AI
#176We get a few thousand fresh grads applying to us each year. It’s practically impossible to interview every one of them. At the same time, any sort of coding assignment we give is easily defeated by AI—so that’s not useful either and there are very few signals there. What we do instead is send out a test - something like a mental ability test - with hundreds of somewhat randomized questions. Many of these are highly v…
We still do a coding assignment, but a significant chunk of the technical interview is dedicated to a walkthrough of the code. Thus far, that’s been able to detect those who relied solely on AI.
…If you used AI and can still explain to me why code works and what it does, even better. You have learned how to use new tools.
(have not tried the randomized question approach to compare, but I’m curious to try it and see what happens)
Re: Interviewing a software engineer who prepared with AI
#177Earlier quoted context omitted.
>When an HVAC company hires a new tech HVAC has certifications you can get. We should strongly consider this in our industry. I don't think its an unreasonable compromise, especially now with the advent of LLMs.
Some parts of the IT industry do lean on certs. IIUC, network engineering in particular is an area where vendor certs play a big role (mainly Cisco). AWS, Azure and GCP all have certs. There are certs for Windows and Linux administration. Java has certs. (I don’t know if anyone cares about the Java certs, but they do exist.)
That is an instructive example.
In regular systems administration, having certs kinda suggested that you didn't have the chops to get a job without a cert. Even people who had them would only include them on the resume when they were explicitly called for in a job description.
With the rise of "DevOps" and throwing half your raise at Amazon, the job moved away from being able to build and run networks of computers. Now it is mostly about configuring off-the-shelf tools in "the cloud". In that world, certs became way more meaningful. Sure, the AWS cert is just testing if you know the six different names Amazon has given one feature, but it is potentially more helpful to know that trivia than it is to actually understand LDAP or DNS.
If AI successfully de-skills software development, maybe certs will finally become useful for developers too.
Re: Interviewing a software engineer who prepared with AI
#178[deleted]
Re: Interviewing a software engineer who prepared with AI
#179I keep coming back to this phrase used in this post: "it was scary". Yeah, hiring is scary. Hiring is insanely expensive on all fronts. Firing people is difficult, it's expensive and legally exposing. Hiring the wrong person, allowing them access your systems and potentially exfiltrate your IP to them is a hazardous but necessary venture. The thing is, none of these things really changed with AI. People have been lyi…
You are right about hiring not being that much different but your prognostications are way off IMO. > - people are probably going to have to fly out for interviews, again. Fly where? Many companies don't have offices anymore. > - awkward and/or neurodivergent people are going to have to learn social skills again. Ahh... the age old, "just do better" position for neurodivergent people. Classic bigotry. > - And yeah, y…
> Classic bigotry.
Hiring is all about finding the best candidate. If you find you cannot function sitting in conference room with three other people for an hour, there is a 100% chance there is a better candidate suited for the role, even if his/her technical skills are less than yours.
Jobs have soft skill requirements, and there is nothing bigoted about that.
Re: Interviewing a software engineer who prepared with AI
#180Earlier quoted context omitted.
Some parts of the IT industry do lean on certs. IIUC, network engineering in particular is an area where vendor certs play a big role (mainly Cisco). AWS, Azure and GCP all have certs. There are certs for Windows and Linux administration. Java has certs. (I don’t know if anyone cares about the Java certs, but they do exist.)
I think in part, the difference in what I mean about certification (perhaps licensure is better word here) is an industry body - accepted and respected generally by the businesses within our industry - that will demonstrate some form of competence
I would love to see a trade union-style group, where you are sponsored to join by an existing member and expected to do some work along side existing members before being certified as journey-level and recommended to employers.
It would require that group to agree on what being a "good" developer meant, but there could be more than one and if you don't agree with this one you could form your own. Maybe one requires people to be able to write testable code and be able to label design patterns, and another expects pure functional programming, and another expects deep security expertise, and companies could know which of those they are looking for and inquire appropriately.
We have this a little bit with employers like Pivotal or ThoughtWorks, that have such strong learning cultures you can be sure that if someone spent five years there they know their stuff. But we could have a version where workers were willing to endorse each other, rather than relying on a specific for-profit company.
It is, like all certifications, only as valuable as the least-competent person who holds it. But the informal versions of this are pretty powerful.