Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Spent 24 Hours with GitHub Copilot Workspaces

every.to

Post reply on HN
Print view View original

31–40 of 76 posts

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#31

Post by frereit »

I'm honestly surprised at the relatively positive reception to this. While there isn't any problem with the code shown, the same effect couldn've probably been achieved with a few well thought out shortcuts in any IDE (delete outerHTML of svg tag, add new tag, add attributes). The only "more complex" output that is shown is the specification that CW produces, which literally contains an error in the first line ("Spral").

Moving on to the complex task, the author simply hand-waves "this isn't good yet but surely it will be". No evidence is given as to _why_ there should be any expectation of LLMs getting there.

And the perceived benefit of discovering that their idea of the more complex task was not thought out enough did not come from the LLM, it came from the author itself. They may as well have spoken to ELIZA or a rubber duck.

What am I missing?

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#32

Post by throwup238 »

HanClinto wrote: post #13
sdesol wrote: post #10

Earlier quoted context omitted.

> then realize that you yourself don't have a perfect idea of what exactly this feature should be I talked about it the last time that Copilot Workspaces reached the front page two days ago and that was, I don't think the value is in the code generation, but rather in the ability to capture our thought process. CW is currently a bottleneck in my opinion and I think the code generation will have to get pretty good bef…

Agreed. The most compelling part of the demo showcased in this post is the way that the tool built the bulleted list of success criteria -- that's so often a tedious and overlooked part of writing user stories, but its importance shouldn't be understated -- the fact that it bakes that step into the workflow feels like the most valuable piece of the puzzle here.

I only got to use the Workspaces feature for an hour before they fixed the waitlist access check but IMO the real value is that it provides a familiar PR-style interface for the whole process that enables fast iterations.

TFA didn't show a screenshot of it but the per file plans and the diffs are side by side on a single screen so you can update the per file plan (adding and removing files as needed) and then "re-roll" the code changes as you go. With the Codespaces feature you can even launch the project and get access to a terminal to run stuff and presumably feed the output back into the plan.

It makes it really easy to spot deficiencies in the code, add comments in the plan, and instantly regenerate the code (well, not instantly, there was a queue when I used it). It was a lot smoother than my experience with Copilot Chat, Aider, and Plandex.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#33

Post by easton »

simonw wrote: post #20
sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My experience so far with LLM generated code is that it tends to be pretty easy to maintain in the future, because it uses obvious code patterns and includes genuinely relevant comments. The trick is to know how to program already, and avoid checking in LLM-generated code unless you completely understand every line. If you don't do that you'll run into the same problems as you would if you hire a contractor to build…

> If you don't do that you'll run into the same problems as you would if you hire a contractor to build your codebase without understanding what they did for you.

I guess the difference is now that the contractor is cheap or free (because it’s a LLM), whereas in the old days you’d either hire a person to do the work and not understand or pick up a book and figure it out yourself (or go to school, or whatever). Figuring it out yourself was often cheaper and then you could understand.

(Not that humans can be replaced by LLM devs yet, or that LLM generated code is necessarily unreadable. It’s usually fine as you say.)

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#34

Post by bunderbunder »

sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My sense, after a year of working at a company with an enterprise Copilot subscription:

If your idea of high-quality code is "follows all the standard clean coding practices, uses design patterns, doesn't do anything Sonarqube would complain about, etc.", then it does a great job.

In terms of more abstract, design-level aspects of code quality, though, I have been less impressed. So, things like limiting statefulness and avoiding unnecessary temporal coupling, good high-level abstractions that obey regular and predictable - ideally algebraic - rules, preservation of well-defined bounded contexts, things like that. Left unchecked, Copilot will happily help you turn a large monolithic codebase into architectural spaghetti.

But then, most humans will do that, too.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#35

Post by siliconc0w »

This debunking video(https://www.youtube.com/watch?v=tNmgmwEtoWE) of Devin really questioned the usefulness for me. It created a file in the repo and spent a lot of time debugging its own unnecessary code rather than reading the Read Me to understand that the code it needed to use already existed and just needed to be run with different inputs.

It's not clear it we're even near a point where it can independently and meaningfully contribute to an existing codebase rather these greenfield demos. Feels similar to the self-driving AI hype where level 5 is still pretty far from realized (Waymo is closest but AIUI still uses a lot of remote human intervention).

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#38

Post by tymscar »

frereit wrote: post #31

I'm honestly surprised at the relatively positive reception to this. While there isn't any problem with the code shown, the same effect couldn've probably been achieved with a few well thought out shortcuts in any IDE (delete outerHTML of svg tag, add new tag, add attributes). The only "more complex" output that is shown is the specification that CW produces, which literally contains an error in the first line ("Sp r…

Youre missing the koolaid. I do wonder if people who cut too much slack to this sort of tech are just doing it because they’re scared of going against the grain. Sort of a vicious cycle.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#39

Post by [deleted] »

sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

[deleted]
Post reply on HN