Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Spent 24 Hours with GitHub Copilot Workspaces

every.to

Post reply on HN
Print view View original

61–70 of 76 posts

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#61

Post by anotherpaulg »

My open source tool aider [0] has long offered an "AI pair programming" workflow that is similar but not identical to Copilot Workspaces.

Aider is more of a collaborative chat, where you work with the LLM interactively asking for a sequence of changes to your git repo. The changes can be non-trivial, modifying a group of files in a coordinated way.

Workspaces seems more agentic. You need to do a bunch of up-front work to (fully) specify the requirements. Even with a perfectly formulated request, agents often go down wrong paths and waste a lot of time and token costs doing the wrong thing.

That's also not how I code personally. My process is usually more iterative.

Another big difference compared to Workspaces is that aider is primarily a CLI tool. Although I just released an experimental browser UI [1] yesterday, making it more approachable for folks who are not fully comfortable on the command line.

[0] https://github.com/paul-gauthier/aider

[1] https://aider.chat/2024/05/02/browser.html

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#62

Post by theshrike79 »

bunderbunder wrote: post #34
sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My sense, after a year of working at a company with an enterprise Copilot subscription: If your idea of high-quality code is "follows all the standard clean coding practices, uses design patterns, doesn't do anything Sonarqube would complain about, etc.", then it does a great job. In terms of more abstract, design-level aspects of code quality, though, I have been less impressed. So, things like limiting statefulness…

Is it verbose? Yes

Will it work? Also yes.

This is usually enough for most cases. Despite HN skewing to the fancier side of programming, the vast majority of day to day programming is just slapping together API glue.

For those cases LLMs like Copilot are excellent. It's a lot faster to ask Copilot about some specific C# thing than start searching through Microsoft's documentation for it. In most cases it can just insert whatever you want at the cursor.

Like just today I pasted a SQL CREATE statement to Copilot and asked it to create a FooModel class of it. Took me 3 seconds of typing, about 5-10 seconds of waiting and clicking "insert at cursor" and I had a 15 property model class.

Repeat a few more times and I've cut down stupid tedious writing by at least 30 minutes and I can go do the more fun bits of attaching some actual logic to those models.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#63

Post by HanClinto »

simonw wrote: post #20
sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My experience so far with LLM generated code is that it tends to be pretty easy to maintain in the future, because it uses obvious code patterns and includes genuinely relevant comments. The trick is to know how to program already, and avoid checking in LLM-generated code unless you completely understand every line. If you don't do that you'll run into the same problems as you would if you hire a contractor to build…

> because it uses obvious code patterns and includes genuinely relevant comments.

I often (simplistically) explain LLMs to people by explaining that it's essentially running a statistical average of language. Next-token-prediction (generally) aims to predict the next-least-surprising word that would occur in a sequence. It aims to "make sense" and be unsurprising.

If you want creative writing and innovative research papers and novel ideas, this isn't going to get you very far.

But if the things you want are "unsurprising" or "predictable" (great attributes of good, maintainable source code), then using this to write code feels like a pretty darn good fit.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#64

Post by sottol »

simonw wrote: post #20
sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My experience so far with LLM generated code is that it tends to be pretty easy to maintain in the future, because it uses obvious code patterns and includes genuinely relevant comments. The trick is to know how to program already, and avoid checking in LLM-generated code unless you completely understand every line. If you don't do that you'll run into the same problems as you would if you hire a contractor to build…

> If you don't do that you'll run into the same problems as you would if you hire a contractor to build your codebase without understanding what they did for you.

I really like this way of thinking about using LLMs, I think that's a great analogy in many ways.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#65

Post by sottol »

easton wrote: post #33
simonw wrote: post #20

Earlier quoted context omitted.

My experience so far with LLM generated code is that it tends to be pretty easy to maintain in the future, because it uses obvious code patterns and includes genuinely relevant comments. The trick is to know how to program already, and avoid checking in LLM-generated code unless you completely understand every line. If you don't do that you'll run into the same problems as you would if you hire a contractor to build…

> If you don't do that you'll run into the same problems as you would if you hire a contractor to build your codebase without understanding what they did for you. I guess the difference is now that the contractor is cheap or free (because it’s a LLM), whereas in the old days you’d either hire a person to do the work and not understand or pick up a book and figure it out yourself (or go to school, or whatever). Figuri…

I have a feeling that if there was not a healthy amount of competition in the space, the prices would start to trend towards the cost of human work.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#66

Post by krainboltgreene »

lordswork wrote: post #30
ossobuco wrote: post #22

Earlier quoted context omitted.

> but if you try that you'll find yourself supplying an enormous amount of context you didn't expect to have to communicate. Context not even directly related to the code at hand, but about the broader business or industry, past lessons learned, something the CEO said to you last week about the feature, etc. Basically you go from programmer to product manager, except you also get to micromanage a non-sentient program…

What prevents an AI agent from becoming the product manager as well, and communicating with you (the customer) to clarify requirements?

Slavery laws, presumably.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#67

Post by ec109685 »

HanClinto wrote: post #8

When reading posts such as these, it occurs to me that AI is increasing the rate / lowering the bar for developers to make the jump to leader / architect. Look at the lessons that the author has learned here: * More specificity == better * The importance of clear bulleted delivery items / criteria-for-success * Unspecified details around a general goal is a ripe area for disappointment All of these are things that a…

This isn’t architect level planning. Even a junior developer should be able to work from vague requirements and build a mental like that.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#68

Post by danpalmer »

bunderbunder wrote: post #34
sottol wrote: post #19

The main thing that makes me skeptical is still what happens to a code base when you do this longer-term. And not just the code base but also the company when nobody understands the code any longer, but maybe neither are problems. A couple questions: * Will the codebase turn into a mess over time by having the AI apply changes over changes over changes? Do we even care? Or do we want a human to still be able to follo…

My sense, after a year of working at a company with an enterprise Copilot subscription: If your idea of high-quality code is "follows all the standard clean coding practices, uses design patterns, doesn't do anything Sonarqube would complain about, etc.", then it does a great job. In terms of more abstract, design-level aspects of code quality, though, I have been less impressed. So, things like limiting statefulness…

Limiting state is the big problem here in my opinion. I’ve also noticed the tendency of AI tools to just add more variables to make things work, which is fine for write-only code, but makes it harder for humans and AI tools to maintain it in the future.

However I think this is also the hard bit for humans to do. It’s one of the most frequent stumbling blocks I see for more junior engineers, and one of the things I notice most when working with code from people who are really good programmers.

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#69

Post by objectivetruth »

anotherpaulg wrote: post #61

My open source tool aider [0] has long offered an "AI pair programming" workflow that is similar but not identical to Copilot Workspaces. Aider is more of a collaborative chat, where you work with the LLM interactively asking for a sequence of changes to your git repo. The changes can be non-trivial, modifying a group of files in a coordinated way. Workspaces seems more agentic. You need to do a bunch of up-front wor…

[dead]

Re: I Spent 24 Hours with GitHub Copilot Workspaces

#70

Post by andrewstuart »

I tried GitHub copilot in vscode. It was immensely frustrating.

The main problem was context. It didn’t seem to know what files to use for our discussion, didn’t listen when I told it, didn’t remember when I told it, had no effective way that I could bring files in and out of the discussion.

All this led to a deeply frustrating session of interaction and frankly I hated it. Easier to use ChatGPT web ui and copy and paste in and out.

GitHub copilot I found better in jetbrains ides. It seemed mostly to know what I was asking about though it’s very long was from being good at managing context.

It’s surprising that after the amount of development they’ve put into copilot it still is so bad at what I’d consider to be barest minimum functionality to integrate into an IDE.

Post reply on HN