Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's original take home assignment open sourced

github.com

Post reply on HN
Print view View original

131–140 of 394 posts

Re: Anthropic's original take home assignment open sourced

#131

Post by saagarjha »

apsurd wrote: post #72
lbreakjai wrote: post #70

I consider myself rather smart and good at what I do. It's nice to have a look at problems like these once in a while, to remind myself of how little I know, and how much closer I am to the average than to the top.

disagree. nobody has a monopoly on what metric makes someone good. I don't understand all this leet code optimization. actually i do understand it, but it's a game that will attract game optimizers. the hot take is, there are other games.

This is explicitly not Leetcode, in fact its goal is to attract optimizers

Re: Anthropic's original take home assignment open sourced

#132

Post by entrox »

lovich wrote: post #84
andruby wrote: post #79

Earlier quoted context omitted.

I understand how it can be interpreted as snarky, but how could it have been written better? It's a hard path to walk and recruiting/interviewing is inherently sensitive it seems.

The original >If you optimize below 1487 cycles, beating Claude Opus 4.5's best performance at launch, email us at performance-recruiting@anthropic.com with your code (and ideally a resume) so we can be appropriately impressed and perhaps discuss interviewing. Not condescending > If you optimize below 1487 cycles, beating Claude Opus 4.5's best performance at launch, email us at performance-recruiting@anthropic.com w…

But now the meaning is different: you went from a potential interview to a guaranteed one.

Re: Anthropic's original take home assignment open sourced

#133

Post by measurablefunc »

saagarjha wrote: post #130
measurablefunc wrote: post #114

Earlier quoted context omitted.

Generate instructions for their simulator to compute some numbers (hashes) in whatever is considered the memory of their "machine"¹. I didn't see any places where they actually disallow cheating b/c it says they only check the final state of the memory² so seems like if you know the final state you could just "load" the final state into memory. The cycle count is supposedly the LLM figuring out the fewest number of i…

Well, they read your code in the actual hiring loop.

My point still stands. I don't know what the LLM is doing so my guess is it's cheating unless there is evidence to the contrary.

Re: Anthropic's original take home assignment open sourced

#134

Post by saagarjha »

wiz21c wrote: post #121
avaer wrote: post #12

It's pretty interesting how close this assignment looks to demoscene [1] golf [2]. [1] https://en.wikipedia.org/wiki/Demoscene [2] https://en.wikipedia.org/wiki/Code_golf It even uses Chrome tracing tools for profiling, which is pretty cool: https://github.com/anthropics/original_performance_takehome/...

I was in the demoscene long ago and that kind of optimisation is definitely in the ballpark of what we did: optimize algorithm down to machine code level (and additionally, cheat like hell to make you believe we ran the algorithm for real :-)). But to be honest, I wonder what algorithm they implement. I have read the code for 2 minutes, and it sound like random forest prediction. Anyone knows what the code does ?

It’s some useless problem like a random tree walk or something like that, the actual algorithm is not particularly important to the problem

Re: Anthropic's original take home assignment open sourced

#135

Post by darkwater »

petters wrote: post #71
bytesandbits wrote: post #60

Having done a bunch of take home for big (and small) AI labs during interviews, this is the 2nd most interesting one I have seen so far.

And the answer to the obvious follow-up question is...?

Maybe it's under NDA :)

Re: Anthropic's original take home assignment open sourced

#136

Post by tuetuopay »

apsurd wrote: post #72
lbreakjai wrote: post #70

I consider myself rather smart and good at what I do. It's nice to have a look at problems like these once in a while, to remind myself of how little I know, and how much closer I am to the average than to the top.

disagree. nobody has a monopoly on what metric makes someone good. I don't understand all this leet code optimization. actually i do understand it, but it's a game that will attract game optimizers. the hot take is, there are other games.

This is the opposite of leet code.

Yes, this applies to some simulated imaginary CPU with an artificial problem. Except that the job asked here is exactly the core of what a performance engineer will do at anthropic: optimize kernels for their fleet of GPUs. Is it simplified? Yes! (e.g. the simulator does not restrict memory access patterns)

This is a real-world problem adapted to a lab setting that can fit in one's head in a matter of hours. Leetcode would have you reimplement the hashmap used in there.

Re: Anthropic's original take home assignment open sourced

#137

Post by saagarjha »

alcasa wrote: post #115
forgotpwd16 wrote: post #82

Earlier quoted context omitted.

Unless misread, 2 hours isn't the time limit for the candidate to do this but the time Claude eventually needed to outperform best returned solution. Best candidate could've taken 6h~2d to achieve this result.

No the 2 hours is their time limit for candidates. The thing is that you are allowed to use any non-human help for their take homes (open book), so if AI can solve it in below 2 hours, it's not very good at assessing the human.

4 hours but AI help is (was?) allowed. I assume it was retired because of Opus basically oneshotting it

Re: Anthropic's original take home assignment open sourced

#139

Post by saagarjha »

Maro wrote: post #39

> This repo contains a version of Anthropic's original performance take-home, before Claude Opus 4.5 started doing better than humans given only 2 hours. Was the screening format here that this problem was sent out, and candidates had to reply with a solution within 2 hours ? Or, are they just saying that the latest frontier coding models do better in 2 hours than human candidates have done in the past in multiple da…

4 hours

Re: Anthropic's original take home assignment open sourced

#140

Post by saagarjha »

benreesman wrote: post #56
greesil wrote: post #14

This is a knowledge test of GPU architecture?

It's a test of polyhedral layout algebra, what NVIDIA calls CuTe and the forthcoming C++ standard calls std::mdspan. This is the general framework for reasoning about correct memory addressing in the presence of arbitrary constraints like those of hardware.

You can get pretty far without needing to care about this fwiw
Post reply on HN