Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's original take home assignment open sourced

github.com

Post reply on HN
Print view View original

311–320 of 394 posts

Re: Anthropic's original take home assignment open sourced

#311

Post by hackpelican »

bjackman wrote: post #181
sva_ wrote: post #174

Earlier quoted context omitted.

I feel like sometimes it just loops those messages when it doesn't actually generate new tokens. But I might be wrong

There are some other failure modes that all feel kinda vaguely related that probably help with building a hypothesis about what's going wrong: Sometimes Gemini tools will just randomly stop and pass the buck back to you. The last thing will be like "I will read the code to understand " and then it waits for another prompt. So I just type "continue" and it starts work again. And, sometimes it will spit out the interna…

At one point it started spitting out its CoT in the comments of the code it’s supposed to be changing.

Re: Anthropic's original take home assignment open sourced

#312

Post by HarHarVeryFunny »

HarHarVeryFunny wrote: post #241
HarHarVeryFunny wrote: post #232

Earlier quoted context omitted.

It does seem a bit of a strange challenge - a bit reminiscent of high school math problems where understanding the question was as much part of it as actually solving the problem when you understood it. Since the focus of the challenge appears(?) intended to be optimization, not reverse engineering, it's a bit odd that they don't give a clear statement of what the kernel is meant to be computing. Perhaps the challeng…

I just threw this prompt at Gemini, and it seems (I haven't analyzed the problem to see if it is correct), to be able to extract a clear understanding of the problem, and a specification for the kernel. "Can you "reverse engineer" what the kernel in this optimization exercise is actually doing - write a specification for it? https://github.com/anthropics/original_performance_takehome " Gemini says it's doing inferenc…

So looking at the actual code (reference_kernel() in problem.py), this "random forest inference" is completely wrong!

It's doing some sort of binary tree traversal, but the hashing and wrap around looks weird - maybe just a made up task rather than any useful algorithm?

Re: Anthropic's original take home assignment open sourced

#313

Post by wiseowise »

jackblemming wrote: post #126
wiseowise wrote: post #88

Earlier quoted context omitted.

Your comment history is littered with “nerds”, “elite”, “better” and all sorts of comparisons. > I was reminding them that there is always someone smarter. And even with this comment you literally do not understand that you have some skewed view of the world. Do you have some high school trauma?

Where I come from, nerd is a term of endearment buddy. > And even with this comment you literally do not understand that you have some skewed view of the world. I’m well aware I don’t have a perfect view of reality and the map isn’t the territory. Do you?

My bad. I jumped on incorrect conclusion. Sorry.

Re: Anthropic's original take home assignment open sourced

#315

Post by ece »

pvalue005 wrote: post #85
bird0861 wrote: post #68

Earlier quoted context omitted.

Which Gemini model did you use? My experience since launch of G3Pro has been that it absolutely sucks dog crap through a coffee straw.

/model: Auto (Gemini 3) Let Gemini CLI decide the best model for the task: gemini-3-pro, gemini-3-flash After ~40 minutes, it got to: The final result is 2799 cycles, a 52x speedup over the baseline. I successfully implemented Register Residency, Loop Unrolling, and optimized Index Updates to achieve this, passing all correctness and baseline speedup tests. While I didn't beat the Opus benchmarks due to the complexit…

After an hour with a few prompts, the first working version got to 3529 cycles (41x speedup) for me. I was using Gemini 3 pro preview.

Re: Anthropic's original take home assignment open sourced

#316

Post by Fwirt »

giancarlostoro wrote: post #229
fny wrote: post #186

Earlier quoted context omitted.

It's because the higher up the stack you go, tools become more declarative and literate. Calling sort is far easier than understanding the algorithm for example.

> Calling sort is far easier than understanding the algorithm for example. This was one of my gripes in college, why am I implementing something if I just need to understand what it does? I'm going to use the built-in version anyway.

My high school computer science teacher (best one I ever had) once told us this anecdote when we were learning sorting algorithms:

He was brought in by the state to do some coaching for existing software devs back in the 90s. When he was going over the various different basic algorithms (insertion sort, selection sort, etc.) one of the devs in the back of the class piped up with, "why are you wasting our time? C++ has qsort built in."

When you're processing millions of records, many of which are probably already sorted, using an insertion sort to put a few new records into a sorted list, or using selection sort to grab the few records you need to the front of the queue, is going to be an order of magnitude faster than just calling qsort every time.

Turned out he worked for department of revenue. So my teacher roasted him with "oh, so you're the reason it takes us so long to get our tax returns back."

Thinking that you can just scoot by using the built-in version is how we get to the horrible state of optimization that we're in. Software has gotten slow because devs have gotten lazy and don't bother to understand the basics of programming anymore. We should be running a machine shop, not trying to build a jet engine out of Lego.

Re: Anthropic's original take home assignment open sourced

#317

Post by vel0city »

HarHarVeryFunny wrote: post #297
owlbite wrote: post #247

Earlier quoted context omitted.

I think calling VLIW "an adandoned design" is somewhat of an exaggeration, such architectures are pretty common for embedded audio processing.

x86-64 SSE and AVX are also SIMD

SIMD and VLIW are somewhat similar but very different in the end.

Re: Anthropic's original take home assignment open sourced

#319

Post by HarHarVeryFunny »

fc417fc802 wrote: post #304
HarHarVeryFunny wrote: post #296

Earlier quoted context omitted.

Well, yes, they have a reference implementation as documentation, just as they have the simulator as documentation for the ISA ... The problem is about pipelining memory loads and ALU operations, so why not just give clear documentatation and state the task rather than "here's a kernel - optimize it"? \_(ツ)_/

Presumably that is only one of two purposes, with the other being to test your ability to efficiently read, understand, and edit low level code that you didn't write. I imagine you'd regularly run into raw PTX if you worked for them in the relevant capacity. And perhaps a third purpose is to use the simulator to test your ability to reason about hardware that you are only just getting familiar with.

I would assume that anyone optimizing kernels at Anthropic has full documentation and specs for what they are working on, as well as a personal butler attending to their every need. This is big money work - every 1% performance improvement must translate to millions of cost savings.

Maybe they specified the challenge in this half-assed way to deliberately test those sorts of skills (even if irrelevant to the job), or maybe it was just lazily put together.

The other thing to note is that if you look at what the reference_kernel() is actually doing, it really looks like a somewhat arbitrary synthetic task (hashes, wraparound), so any accurate task specification would really need to be a "line by line" description of the steps, at which point you may as well just say "here's some code - do this".

Re: Anthropic's original take home assignment open sourced

#320

Post by sigbottle »

ActorNightly wrote: post #310
lbreakjai wrote: post #70

I consider myself rather smart and good at what I do. It's nice to have a look at problems like these once in a while, to remind myself of how little I know, and how much closer I am to the average than to the top.

Yours is a good mentality to have because it creates the emotional drive to learn more, so don't lose that. That being said, this isn't really that complicated. Its just a matter of taking enough time to look at the code and understand how its structured. I feel like the thing that differentiates developer skill is pretty much being able to do that, specifically in the process of having the model of the program in yo…

Does it?

For me, I've had that mentality for the longest time and I didn't get anything done because, well, "I'm just average".

For me, a little bit of arrogance (there's no way I couldn't do X, let's go do it), even if I end up "looking stupid" (see, I told you it was that hard!), was far more valuable to my development

Post reply on HN