[flagged]
Asana cleared 5 years of engineering work in 2 weeks with Codex
31–40 of 103 posts
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#32> An engineer checked progress twice a day and reviewed every proposed change. It doesn’t sound like that much changed. If one engineer can really review all the changes in under two weeks, the original estimate of 5 years of engineering work has to be waaaaaaay off.
A nitpick. While this definitely does not apply to the problem described, some changes are indeed mostly alignment, not implementation.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#33From the article: > For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M. If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
The LLM is reasoning about estimates from its training data... which is to say, from human engineering timescales.
I suspect the labs could improve the models such that they are estimating these sorts of things but they don't prioritize doing so (or perhaps RLHF selects it away) because, as you say, it feels amazing to do a week's worth of work in an hour.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#34Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work. Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it…
> people will work very hard in the background to fix it, with no fanfare.
This. It looks like AI companies have managed to use this for their advantage. Can't really blame them.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#35Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work. Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it…
> where afraid to touch for good reasons This appears entirely unreasonable. Normally the reason is not good. The reason is that unit testing is missing or that downstream effects are not entirely mapped out. Exactly activities that traditional software developers are loathing because they are boring and mentally straining.
Or the reason is that something is just unfeasible with the existing architecture, or the reason is that building this would break important technical assumptions... All kinds of things.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#36It is a good PR for Asana. I checked this with Gemini & here is what Gemini said - https://share.gemini.google/B8nocObwgiih Summary: The Takeaway Did an AI generate 5 years' worth of highly creative, novel software features from scratch in two weeks? No. Did an AI complete 5 years' worth of tedious, widespread code-migration technical debt in two weeks? Yes.
These conclusion can be obtained by any individual adult with a command of the English language and average written text comprehension abilities within 2 minutes. That’s how long it takes to read and understand the case study.
So, why did you surrender your cognition to an LLM? Do you have some form of mental impairment?
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#37If you ask Sol or Claude how much time it will take to implement a plan they just came up with, they usually advise a timeframe in the weeks or months - assuming, I suppose, that human programmers will be building it. And then you ask the model to just "do it" and it takes an hour or two. I always find this entertaining.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#38Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work. Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people were afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will work very hard in the background to fix it…
> I'm not sure why these types of news are still coming out when we all have AI at work. Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
I can for sure say that this kind of statement is exaggeration if not a blatant lie. It may work for PR but it gives a wrong impression/ideas. Now engineers should be ready to be questioned by their management for all the rewrites that they have been estimating in months if not years.
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#39"An engineer checked progress twice a day and reviewed every proposed change. Simpler instructions worked better than a more elaborate setup."
Oh dear, poor Asana customers...
Re: Asana cleared 5 years of engineering work in 2 weeks with Codex
#40I don't trust anything OpenAI says anymore.