Live data from Hacker News

Anthropic's original take home assignment open sourced

github.com

281–290 of 394 posts

Re: Anthropic's original take home assignment open sourced

#281

> If you optimize below 1487 cycles, beating Claude Opus 4.5's best performance at launch, email us at performance-recruiting@anthropic.com with your code (and ideally a resume) so we can be appropriately impressed and perhaps discuss interviewing. This is an interesting way to recruit. Much better than standard 2 leetcode medium/hard questions in 45 mins.

It would take something like one week full time to work on this. It's not something you can do if you have a full-time job and apply to several other companies. I find it unreasonable to ask a candidate to spend that much time for an uncertain result.

It's true that being ready for leetcode takes practice, but at least it's standard so you can re-use the skills to other interviews. Optimizing some generated code is certainly fun, but it's as useless as leetcode for your average programmer.

Re: Anthropic's original take home assignment open sourced

#282
post #274

Earlier quoted context omitted.

One could use any number of LLMs on real-world problems. Why are we still interviewing like its 1999?

Old habits die hard. And engineers are pretty lazy when it comes to interviews, so just throwing the same leetcode problem into coder pad in every interview makes interviews easier for the person doing the interview.

If you want people to interview better, you have to both allocate resources to it, and make it count on perf. It’s not laziness, it’s ROI.

Re: Anthropic's original take home assignment open sourced

#283
post #274

Earlier quoted context omitted.

One could use any number of LLMs on real-world problems. Why are we still interviewing like its 1999?

Old habits die hard. And engineers are pretty lazy when it comes to interviews, so just throwing the same leetcode problem into coder pad in every interview makes interviews easier for the person doing the interview.

As an interviewer, I ask the same problems because it makes it much easier to compare candidates.

Re: Anthropic's original take home assignment open sourced

#284

Earlier quoted context omitted.

Old habits die hard. And engineers are pretty lazy when it comes to interviews, so just throwing the same leetcode problem into coder pad in every interview makes interviews easier for the person doing the interview.

As an interviewer, I ask the same problems because it makes it much easier to compare candidates.

How do you know if one candidate happened to see the problem on leetcode and memorized the solution versus one who struggled but figured it out slower?

Re: Anthropic's original take home assignment open sourced

#285

Earlier quoted context omitted.

It does seem a bit of a strange challenge - a bit reminiscent of high school math problems where understanding the question was as much part of it as actually solving the problem when you understood it. Since the focus of the challenge appears(?) intended to be optimization, not reverse engineering, it's a bit odd that they don't give a clear statement of what the kernel is meant to be computing. Perhaps the challeng…

This isn't "reverse engineering" it's merely "being able to read fairly simple code you didn't write". A much simpler version of the kernel is provided at the end of problem.py as reference_kernel2. If you can't make sense of such a small codebase or don't immediately recognize the algorithm that's being used (I'm guilty of the latter) then you presumably aren't someone that they want to hire.

Fair enough, and there are clues in the comments too, but why not just provide the specification of the kernel (inputs and outputs) as part of the problem?

Re: Anthropic's original take home assignment open sourced

#286

Earlier quoted context omitted.

As an interviewer, I ask the same problems because it makes it much easier to compare candidates.

How do you know if one candidate happened to see the problem on leetcode and memorized the solution versus one who struggled but figured it out slower?

It doesn’t matter. It’s about looking for candidates who have put in the time for your stupid hazing ritual. It signals on people who are willing to dedicate a lot of time to meaningless endeavors for the sake of employment.

This type of individual is more likely to follow orders and work hard - and most importantly - be like the other employees you hired.

Re: Anthropic's original take home assignment open sourced

#287

> If you optimize below 1487 cycles, beating Claude Opus 4.5's best performance at launch, email us at performance-recruiting@anthropic.com with your code (and ideally a resume) so we can be appropriately impressed and perhaps discuss interviewing. This is an interesting way to recruit. Much better than standard 2 leetcode medium/hard questions in 45 mins.

It would take something like one week full time to work on this. It's not something you can do if you have a full-time job and apply to several other companies. I find it unreasonable to ask a candidate to spend that much time for an uncertain result. It's true that being ready for leetcode takes practice, but at least it's standard so you can re-use the skills to other interviews. Optimizing some generated code is c…

As long as there are qualified candidates willing to do unreasonable tasks for the chance to work at a company, there's not much incentive for the company to change their system. Those people will also probably work unreasonably hard and make unreasonable sacrifices for the company.

Re: Anthropic's original take home assignment open sourced

#288

Earlier quoted context omitted.

This isn't "reverse engineering" it's merely "being able to read fairly simple code you didn't write". A much simpler version of the kernel is provided at the end of problem.py as reference_kernel2. If you can't make sense of such a small codebase or don't immediately recognize the algorithm that's being used (I'm guilty of the latter) then you presumably aren't someone that they want to hire.

Fair enough, and there are clues in the comments too, but why not just provide the specification of the kernel (inputs and outputs) as part of the problem?

They do. They provide reference_kernel which shows the algorithm itself, build_mem_image which shows the data format you will be working with, and finally reference_kernel2 which implements said algorithm on said data format.

They then provide you with a very naive implementation that runs on their (very simple) VLIW architecture that you are to optimize.

If at the end of that someone is still lost I think it is safe to say it was their goal that person should fail.

Re: Anthropic's original take home assignment open sourced

#289

Earlier quoted context omitted.

The question isn't clearly written down anywhere, that's why. Presumably actual candidates would have been given more info over the phone or email. Part of the "challenge" is reverse engineering their Python; unclear if that's intentional. If you look at the top of perf_takehome.py then there is a brief comment saying the challenge is to optimize a kernel. Kernel in GPU land means a program that computes on data in p…

I think calling VLIW "an adandoned design" is somewhat of an exaggeration, such architectures are pretty common for embedded audio processing.

Worth adding on that note:

From JAX to VLIW: Tracing a Computation Through the TPU Compiler Stack, https://patricktoulme.substack.com/p/from-jax-to-vliw-tracin...

Google’s Training Chips Revealed: TPUv2 and TPUv3, HotChips 2020, https://hc32.hotchips.org/assets/program/conference/day2/Hot...

Ten Lessons From Three Generations Shaped Google’s TPUv4i, ISCA 2021, https://gwern.net/doc/ai/scaling/hardware/2021-jouppi.pdf

Post reply on HN