Can LLMs earn $1M from real freelance coding work?
newsletter.getdx.com
Can LLMs earn $1M from real freelance coding work?
1–10 of 19 posts
Re: Can LLMs earn $1M from real freelance coding work?
#2This resonated with me based on my recent experience using Claude to help me code. I almost gave up, but re-phrased the initial request (after 7-10 failed tries) and it finally nailed it.
> 3. Performance improves with multiple attempts Allowing the o1 model 7 attempts instead of 1 nearly tripled its success rate, going from 16.5% to 46.5%. This hints that current models may have the knowledge to solve many more problems but struggle with execution on the first try.
https://newsletter.getdx.com/i/160797867/performance-improve...
Re: Can LLMs earn $1M from real freelance coding work?
#3tl;dr, and as Betteridge's Law would lead you to believe, the answer is no.
Re: Can LLMs earn $1M from real freelance coding work?
#4How do they know the tasks were "solved"? Wouldn't that require the customer to be happy, and pay the bounty?
Re: Can LLMs earn $1M from real freelance coding work?
#5This resonated with me based on my recent experience using Claude to help me code. I almost gave up, but re-phrased the initial request (after 7-10 failed tries) and it finally nailed it. > 3. Performance improves with multiple attempts Allowing the o1 model 7 attempts instead of 1 nearly tripled its success rate, going from 16.5% to 46.5%. This hints that current models may have the knowledge to solve many more prob…
I haven't really messed with Claude or other programming AIs much, but when using chatgpt for random stuff, it seems like the safety rails end up blocking a lot of stuff and rephrasing to get around them is necessary. I wonder if some of these programming AIs would be more useful if it some of the context that causes them to produce invalid results was more obvious to the users.
Re: Can LLMs earn $1M from real freelance coding work?
#6tl;dr, and as Betteridge's Law would lead you to believe, the answer is no.
>Betteridge's Law
Is that the one that says if an article title ends in a question that means the answer is no?
Re: Can LLMs earn $1M from real freelance coding work?
#7Unless I am repeatedly missing it, it’s not mentioned in the article how much money the researchers spent performing the tests. What was the budget for the AI execution? If the researchers only spent $10,000 to “earn” $400,000, that’s amazing, whereas if they spent $500,000 for the same result, that’s obviously less exciting.
Re: Can LLMs earn $1M from real freelance coding work?
#8There goes all the low-hanging fruit ...
Re: Can LLMs earn $1M from real freelance coding work?
#9This resonated with me based on my recent experience using Claude to help me code. I almost gave up, but re-phrased the initial request (after 7-10 failed tries) and it finally nailed it. > 3. Performance improves with multiple attempts Allowing the o1 model 7 attempts instead of 1 nearly tripled its success rate, going from 16.5% to 46.5%. This hints that current models may have the knowledge to solve many more prob…
I haven't really messed with Claude or other programming AIs much, but when using chatgpt for random stuff, it seems like the safety rails end up blocking a lot of stuff and rephrasing to get around them is necessary. I wonder if some of these programming AIs would be more useful if it some of the context that causes them to produce invalid results was more obvious to the users.
> safety rails end up blocking a lot of stuff
curious if you had any examples. i'm fairly meh on llm coding myself but have a pet theory on safety rails. i've certainly hit plenty myself but not with coding with llm's.
Re: Can LLMs earn $1M from real freelance coding work?
#10Honestly, this reads like an AI-generated summary.
Discussion on original paper: https://news.ycombinator.com/item?id=43086347