Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's original take home assignment open sourced

github.com

Post reply on HN
Print view View original

341–350 of 394 posts

Re: Anthropic's original take home assignment open sourced

#341

Post by whateveracct »

scottyah wrote: post #332
whateveracct wrote: post #264

Earlier quoted context omitted.

damn that sucks i guess that ensures you either hire the childless or those with children who are fine with be not present for that long willingly (so they are probably gonna be job-obsessed enough) or they are currently unemployed so they won't have an existing job as anchoring leverage well played, anthropic

I can't imagine wanting to hire someone as an FTE who is unable to spend 4hrs working in a day.

i can't but i put out staff level work and get paid happily accordingly for years now

nobody i know ever spends 4hrs uninterrupted working remotely lolol

Re: Anthropic's original take home assignment open sourced

#342

Post by LarsKrimi »

I liked the core challenge. Finding the balance of ALU and VALU, but I think that the problem with the load bandwidth could lead to problems

Like optimizing for people who assume the start indices always will be zero. I am close to 100% sure that's required to get below 2096 total loads but it's just not fun

If it however had some kind of dynamic vector lane rotate that could have been way more interesting

Re: Anthropic's original take home assignment open sourced

#343

Post by red75prime »

measurablefunc wrote: post #269
red75prime wrote: post #191

Earlier quoted context omitted.

And? Anthropic is not aware of this 2020 paper? The problem is not solvable?

Why are you asking me? Email & ask Anthropic.

Obviously, because you use this old paper as an argument.

Re: Anthropic's original take home assignment open sourced

#344

Post by measurablefunc »

red75prime wrote: post #343
measurablefunc wrote: post #269

Earlier quoted context omitted.

Why are you asking me? Email & ask Anthropic.

Obviously, because you use this old paper as an argument.

I don't have any insider information on what they know or don't know so you're welcome to keep asking nonsensical questions but eventually I'll stop answering.

Re: Anthropic's original take home assignment open sourced

#345

Post by a24j »

languid-photic wrote: post #226
a24j wrote: post #120

Earlier quoted context omitted.

Can you share the agent-comparison harness code or point to something similar? I want to learn about benchmarking models in a basic or practical sense.

Sure! https://github.com/voratiq/voratiq

Thanks so much!!

Re: Anthropic's original take home assignment open sourced

#346

Post by cgearhart »

NitpickLawyer wrote: post #32

The writing was on the wall for about half a year (publicly) now. The oAI 2nd place at the atcoder world championship competition was the first one, and I remember it being dismissed at the time. Sakana also got 1st place in another atcoder competition a few weeks ago. Google also released a blog a few months back on gemini 2.5 netting them 1% reduction in training time on real-world tasks by optimising kernels. If t…

I think this is the actual “bitter lesson”—the scalable solution (letting LLMs bang against the problem nonstop) will eventually far outperform human effort. There will come a point—whether sooner or later—where this’ll be the expected norm for handling such problems. I think the only question is whether there is any distinction between problems like this (clearly defined with a verifiable outcome) vs the space of all interesting computer programs. (At the moment I think there’s space between them. TBD.)

Re: Anthropic's original take home assignment open sourced

#347

Post by johnnyanmac »

TrackerFF wrote: post #95
lbreakjai wrote: post #70

I consider myself rather smart and good at what I do. It's nice to have a look at problems like these once in a while, to remind myself of how little I know, and how much closer I am to the average than to the top.

Well it is a specialized problem. If you've never worked on anything similar previously, it is going to take time. Don't even need to interview for selective billion dollar companies like Anthropic to encounter these types of problems - after college I interviewed for various electronics/hardware companies where you'd get asked to optimize low-level code - which would have looked quite foreign, if you had never actua…

>Don't even need to interview for selective billion dollar companies like Anthropic to encounter these types of problems

I'll take any interviews at this point in time.

But yes, every domain has its jargon. I work tangentially to this and quickly understood this as a GPGPU problem. A relatively elementary one if you studied this space, though a time limit of 2 hours seems overly restrictive if you aren't actively studying this stuff.

Re: Anthropic's original take home assignment open sourced

#348

Post by johnnyanmac »

sublinear wrote: post #246
Onavo wrote: post #117

Earlier quoted context omitted.

If you ask an EE to debug react state management code without prior exposure they won't do too well either. But on the other hand they can easily pick up most of it after a week long crash course while training a performance engineer who can optimize code for a specific architecture would take months.

> they can easily pick up most of it after a week long crash course I have to disagree and question what you mean by "optimization". It's very easy to write web code that technically accomplishes a task, but does so poorly. This is the natural consequence of having so many options available. The vast majority of web devs with less than 5 years of experience simply don't understand plain javascript well enough. It's a…

>The vast majority of web devs with less than 5 years of experience simply don't understand plain javascript well enough

they are never tested on it, and many won't dig that deep in the day-to-day. Whose fault is it that they don't know plain javascript well enough? That's the result of shipping "content" over any other metric of proper software engineering.

Funnily enough I did take a mini-course (not a week, but we're talking maybe 100 hours of work as a recreational online summer class) in plain javascript at my university. Quite the quirky language. But this was in ES3 or so, so maybe there's many more guard rails these days against the core jank that makes up JS

Re: Anthropic's original take home assignment open sourced

#349

Post by johnnyanmac »

giancarlostoro wrote: post #229
fny wrote: post #186

Earlier quoted context omitted.

It's because the higher up the stack you go, tools become more declarative and literate. Calling sort is far easier than understanding the algorithm for example.

> Calling sort is far easier than understanding the algorithm for example. This was one of my gripes in college, why am I implementing something if I just need to understand what it does? I'm going to use the built-in version anyway.

I mean, the lesson I got from my 10X class was pretty much that: "never write your own math library, unless you're working on maintaining one yourself".

funnily enough, this wasn't limited to contributing to some popular OS initiative. You can call YAGNI, but many companies do in fact have their own libraries to maintain internally. So it comes up more than you expect.

On a higher level, the time I took to implement a bunch of sorts helped me be able to read the docs for sort(), realize it's a quicksort implentation, and make judgements like

1. yeah, that works

2. this is overkill for my small dataset, I'll just whip up basic bubblesort

3. oh, there's multiple sort API's and some sorts are in-place. I'll use this one

4. This is an important operation and I need a more robust sorting library. I'll explain it to the team with XYZ

The reasoning was the important lesson, not the ability to know what sorting is.

Re: Anthropic's original take home assignment open sourced

#350

Post by johnnyanmac »

scottyah wrote: post #295
WithinReason wrote: post #227

Earlier quoted context omitted.

This also shows that a performance engineer's job, even at Anthropic, is to be a glorified human compiler, who is often easily beaten by LLMs.

I think the job is to be one of the few that's better than LLMs.

And how would one do that these days if they didn't spend their career doing this pre-LLM? Just expect to study and perform such projects as a hobby for a few years on the side? These are specialized problems that you only really do for a few select companies.
Post reply on HN