Earlier quoted context omitted.
I read it, you're mistaken.
I did the assignment my guy
That's great but I didn't ask & that's still not addressing my point.
Anthropic's original take home assignment open sourced
321–330 of 394 posts
Re: Anthropic's original take home assignment open sourced
#322Earlier quoted context omitted.
This has nothing to do w/ me & consistently making it a personal problem instead of addressing the claims is a common tactic for people who do not know what it means to present evidence for their claims. Anthropic has not provided the necessary evidence for me to conclude that their LLM is not cheating. I have no opinion on their competence b/c that is not what is at issue. They could be incompetent & not notice that…
You are implying that you believe them to be incompetent since otherwise you would not expect evidence in this instance. They also haven't provided independent verification of their claims - do you suspect them of lying as well? How do you explain the specific score that was achieved if as you suggest the LLM simply copied the answer directly?
Either they have proof that their LLM is not cheating or they don't. The linked post does not provide evidence that the LLM is not cheating. I don't have to explain anything on my end b/c my claim is very simple & easily refuted w/ the proper evidence.
Re: Anthropic's original take home assignment open sourced
#323Re: Anthropic's original take home assignment open sourced
#324Earlier quoted context omitted.
Presumably that is only one of two purposes, with the other being to test your ability to efficiently read, understand, and edit low level code that you didn't write. I imagine you'd regularly run into raw PTX if you worked for them in the relevant capacity. And perhaps a third purpose is to use the simulator to test your ability to reason about hardware that you are only just getting familiar with.
I would assume that anyone optimizing kernels at Anthropic has full documentation and specs for what they are working on, as well as a personal butler attending to their every need. This is big money work - every 1% performance improvement must translate to millions of cost savings. Maybe they specified the challenge in this half-assed way to deliberately test those sorts of skills (even if irrelevant to the job), or…
They do, but documentation is not always complete or correct.
Re: Anthropic's original take home assignment open sourced
#325Earlier quoted context omitted.
I just threw this prompt at Gemini, and it seems (I haven't analyzed the problem to see if it is correct), to be able to extract a clear understanding of the problem, and a specification for the kernel. "Can you "reverse engineer" what the kernel in this optimization exercise is actually doing - write a specification for it? https://github.com/anthropics/original_performance_takehome " Gemini says it's doing inferenc…
So looking at the actual code (reference_kernel() in problem.py), this "random forest inference" is completely wrong! It's doing some sort of binary tree traversal, but the hashing and wrap around looks weird - maybe just a made up task rather than any useful algorithm?
Yes, it’s made up.
Re: Anthropic's original take home assignment open sourced
#326This is a kind of task that's best solved by possibly spending more than the allocated 2 hours on it, once any obvious low-hanging fruit is picked. An optimization task is what a machine does best. So the real problem would be to construct a machine that would be able to run the optimization. A right optimization framework that results from the effort could also efficiently solve many more similar problems in the fut…
> So the real problem would be to construct a machine that would be able to run the optimization.
This is a valid way to solve the problem.
Re: Anthropic's original take home assignment open sourced
#327Earlier quoted context omitted.
I did the assignment my guy
That's great but I didn't ask & that's still not addressing my point.
I didn’t ask you to be rude or wrong either, yet here we are. The assignment is explicitly single core and cycle accurate. Your point is completely irrelevant and shows a disconnect with the content being discussed.
Re: Anthropic's original take home assignment open sourced
#328Earlier quoted context omitted.
Continuous
damn that sucks i guess that ensures you either hire the childless or those with children who are fine with be not present for that long willingly (so they are probably gonna be job-obsessed enough) or they are currently unemployed so they won't have an existing job as anchoring leverage well played, anthropic
I’m trying to imagine what would make it impossible to not pay attention to your children for four hours and the only thing I can think of that can’t be scheduled around is…a very young newborn, maybe? If they’re prone to waking up constantly?
Re: Anthropic's original take home assignment open sourced
#329Earlier quoted context omitted.
damn that sucks i guess that ensures you either hire the childless or those with children who are fine with be not present for that long willingly (so they are probably gonna be job-obsessed enough) or they are currently unemployed so they won't have an existing job as anchoring leverage well played, anthropic
I’m trying to imagine what would make it impossible to not pay attention to your children for four hours and the only thing I can think of that can’t be scheduled around is…a very young newborn, maybe? If they’re prone to waking up constantly?
Babies and toddlers need parental care and attention too.
Re: Anthropic's original take home assignment open sourced
#330Earlier quoted context omitted.
That's great but I didn't ask & that's still not addressing my point.
I didn’t ask you to be rude or wrong either, yet here we are. The assignment is explicitly single core and cycle accurate. Your point is completely irrelevant and shows a disconnect with the content being discussed.
It's neither rude nor wrong to ask for evidence to support claims being made in what appears to be corporate advertising. The claim is their LLM is better than a person, I asked for evidence. None was presented. It's not complicated.