Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Workspace: Technical Preview

github.blog

Post reply on HN
Print view View original

171–180 of 333 posts

Re: GitHub Copilot Workspace: Technical Preview

#171

Post by fwip »

candiddevmike wrote: post #150
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

One thing that's helped me a little bit, is to open up the spec as context, and then asking the LLM to generate tests based on that spec.

Re: GitHub Copilot Workspace: Technical Preview

#172

Post by mucle6 »

Bjorkbat wrote: post #168

You can see my earlier comment somewhere here, but I feel obliged to remind everyone that the claim that Github Copilot made developers "55% more productive" came from a study where they asked 100 developers to implement an HTTP server in Javascript, split the group up roughly 50/50, and gave one group Github Copilot. The Copilot group did it in an hour and 11 minutes, whereas the control group got it done 2 hours an…

Do you not have a good experience with github copilot? I recently started using it and it will write functions that are 90% right based on comments, and I just modify what I want.

I've felt at least a 2x speedup, maybe even 4x. That said I'm working on a new project where I'm writing a lot of code. For making small changes I could see how its much less valuable.

Re: GitHub Copilot Workspace: Technical Preview

#173

Post by nyarlathotep_ »

mike_hearn wrote: post #25
studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

Whoa, don't quit your course because of a product announcement! That'd be overreacting by a lot. Please consider these points instead! Firstly, it's not true that LLMs can structure and execute large scale changes in entire repositories. If you find one that can do that please let me know, because we're all waiting. If you're thinking of the Devin demo, it turned out on close inspection to be not entirely what it see…

> Secondly, do not underestimate the demand for software. For as long as I've been alive, the demand for software has radically outstripped supply. GitHub claims there are now more than 100 million developers in the world.

And yet jobs are more difficult to come by than any time in recent history (regardless of skill or experience; excepting perhaps "muh AI" related roles), a seemingly universally expressed sentiment around these parts.

Re: GitHub Copilot Workspace: Technical Preview

#174

Post by huygens6363 »

studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

It is hard to find direct comparisons as the tech is truly novel, but I have heard people say we don’t need to learn math because your calculator can do “university level math”. I don’t know how close that argument is to yours, but there is some overlap.

Your calculator can indeed do fancy math, but you will not be able to do anything with it because you do not understand it.

This is like fancying yourself an engineer because you constructed an IKEA cupboard or an automotive expert because you watched Youtube.

Anything an amateur can come up with is blown to pieces by an actual expert in a fraction of the time and will be of considerable higher quality. The market will go through a period of adjustment as indeed the easy jobs will be automated, but that makes the hard jobs even harder, not easier.

Once you automate the easy stuff, the hard stuff remains.

Basically:

Expert + AI > Amateur + AI

Re: GitHub Copilot Workspace: Technical Preview

#175

Post by danenania »

Bnjoroge wrote: post #144
danenania wrote: post #95

For anyone who might be interested in an open source, terminal-based approach to using AI on larger tasks and real-world projects, I'm building Plandex: https://github.com/plandex-ai/plandex I've tried to create a very tight feedback loop between the developer and the LLM. I wanted something that feels similar to git. Apart from the planning and code generation itself, Plandex is also strongly focused on version cont…

just played with it. pretty solid! questions for me, how do you validate the new changes will work without any issues after applying them? the biggest issue I have with most code generators is the feedback loop from suggested code -> testing it out -> failing, doing it again. would be great if this was more seamless. additionally, would be helpful to control the code generation process in real-time.

Thanks for trying it! So far, the the testing and validation stage has been left to the developer. While this could change in the future, my experience has been the models aren't quite good enough yet to make an auto-test/auto-fix loop like you've described the default behavior. You end up with a lot of tire-spinning where you're burning a lot of tokens to fix issues that a human can resolve trivially.

I think it's better for now to use LLMs to generate the bulk of a task, then have the developer clean up and integrate rather than trying to get the LLM to do 100%.

That said, you can accomplish a workflow like this with Plandex already by piping output into context. It would look something like:

  plandex new
  plandex load relevant_context.ts some_more_context.ts
  plandex tell 'some kind of complex task'
  # ...Plandex does its thing, but doesn't get it 100% right
  npm test | plandex load
  plandex tell 'please fix the problems causing the failed tests'
As the models improve, I'm definitely interested in baking this in to make it more automated.

Re: GitHub Copilot Workspace: Technical Preview

#176

Post by wiredfool »

studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

Maybe it would help me, not sure. I haven't been impressed with what I've seen when team members have "run stuff through chatgpt". (I'm senior, been doing this professionally for 25 years. Made my share of mistakes, supported my code for a decade or two.)

My main issue at the moment with Junior devs is getting stuck in the weeds, chasing what they think is a syntax error, but not seeing (or hearing) that what they have is a lack of understanding. Some of that is experience, some of that is probably not being able to read the code and internalize what it all means, or make good test cases to exercise it.

If you can't produce the code, and have a sketchy grasp of reasoning it out, debugging it is going to be a step too far. And the AIs are (hopefully) going to be giving you things that look right, there will be subtle bugs. This puts it in the dangerous quadrant.

Re: GitHub Copilot Workspace: Technical Preview

#178

Post by fragmede »

candiddevmike wrote: post #150
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

The skill in using an LLM currently is in getting you to where you want to be, rather than wasting time convincing the LLM to spit out exactly what you want. That means flipping between having Aider write the code and editing the code yourself, when it's clear the LLM doesn't get it, or you get it better than it does.

Re: GitHub Copilot Workspace: Technical Preview

#179

Post by konschubert »

PoignardAzur wrote: post #51

I think Microsoft is going the wrong direction with Copilot (though it's a reasonable direction given their incentives). Right now Copilot is terrible at large changes to complex codebases; the larger and more complex, the worse. But it's great at suggesting very short snippets that guess exactly what you were in the middle of writing and write it for you. I wish Copilot focused more on the user experience at the sma…

Totally.

Re: GitHub Copilot Workspace: Technical Preview

#180

Post by Terretta »

msoad wrote: post #78

This should've been done in VSCode. Most developers don't want to write code in a browser. Once VSCode extension is built they could generalize it to other editors. I'm sure they are thinking of VSCode integration and I am hopping for that to be available soon

But, VSCode is in a browser?

https://vscode.dev/

Post reply on HN