You can try out GitHub Copilot Workspace now in read-only mode:
https://copilot-workspace.githubnext.com/AnandChowdhary/anan...
GitHub Copilot Workspace: Technical Preview
201–210 of 333 posts
Re: GitHub Copilot Workspace: Technical Preview
#202I use Github co-pilot but 55% productive is a bullshit number. Perhaps 1% may be. Most of co-pilot suggestions are either simple pattern matches or subtle hallucinations where I have to catch and fix silly bugs. Github Chat is not very useful at understanding what the code is doing. Tried it once or twice and gave up. The hype will help with Microsoft Stock though. Seems like bean counter management is taking over.
I was actually about to make a comment on that. They got that number from some dumb "experiment" they did where told two groups of developers to implement an HTTP server in Javascript. The group with Copilot got the job done 55% faster ( https://github.blog/2022-09-07-research-quantifying-github-c... ) So yeah, they did an experiment with <100 developers asking them to create implement something that only took the co…
[deleted]
Re: GitHub Copilot Workspace: Technical Preview
#203Earlier quoted context omitted.
Accountants thought spreadsheets would kill their profession, instead demand for them exploded. Compilers made it much easier to code compared to writing everything in Assembly. Python made it much easier to code than writing C. Both increased the demand for coders. Code is a liability, not an asset. The fact that less technical people and people who are not trained engineers can now make useful apps by generating mi…
I've seen it already. A small business owner (one man show) friend of mine with zero developer experience was able to solve his problem (very custom business specific data -> calendar management) in a rough way using ChatGPT. But it got past about 300 lines long and really started to get bad. He'd put dozens of hours of time on his weekend to getting it to where it was by using ChatGPT over and over, but eventually i…
> as the tech matures.
Then it means you can use the matured tech and build in one day a superb service. And improve it the next day.
Re: GitHub Copilot Workspace: Technical Preview
#204Earlier quoted context omitted.
> And I suspect that kind of story will continue quite a bit as the tech matures. Don't you think that this tech can only get better? And that there will come a time in the very near future when the programming capabilities of AI improve substantially over what they are now? After all, AI writing 300 line programs was unheard of a mere 2 years ago. This is what I think GP is ignoring. Spreadsheets couldn't to do ever…
> But AI will continually improve There is a bit of a fallacy in here. We don’t know how far it will improve, and in what ways. Progress isn’t continuous and linear, it comes more in sudden jumps and phases, and often plateaus for quite a while.
Fair enough. But it's enough of a fallacy as asserting that it won't improve, no?
The rate of improvement in the last 5 years hasn't stopped, and in fact has accelerated in the last two. There is some concern that it's slowing down as of 2024, but there is a historically high amount of interest, research, development and investment pouring into the field that it's more reasonable to expect further breakthroughs than not.
If nothing else, we haven't exhausted the improvements from just throwing more compute at existing approaches, so even if the field remains frozen, we are likely to see a few more generational leaps still.
Re: GitHub Copilot Workspace: Technical Preview
#205Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…
Don't forget that this is marketing.
"You'll be the best cook if you buy the Mega Master Automated Kitchen Appliance (with two knives included)"
That line is marketed at me, who does not know how to cook, they're telling me I'll be almost a chef.
You'll hear Jensen say that coding is now an obsolete skill, because he's marketing the capabilities of his products to shareholders, to the press.
It might well be that in 10 years these LLMs are capable of doing really serious stuff, but if you're studying CS now, this would mean for you that in 10 years you'll be able to use these tools much better than someone who will just play with it. You'll really be able to make them work for you.
Re: GitHub Copilot Workspace: Technical Preview
#206Earlier quoted context omitted.
just played with it. pretty solid! questions for me, how do you validate the new changes will work without any issues after applying them? the biggest issue I have with most code generators is the feedback loop from suggested code -> testing it out -> failing, doing it again. would be great if this was more seamless. additionally, would be helpful to control the code generation process in real-time.
Thanks for trying it! So far, the the testing and validation stage has been left to the developer. While this could change in the future, my experience has been the models aren't quite good enough yet to make an auto-test/auto-fix loop like you've described the default behavior. You end up with a lot of tire-spinning where you're burning a lot of tokens to fix issues that a human can resolve trivially. I think it's b…
Out of interest, what kind of cost ranges are you seeing users spend on the OpenAI API using Plandex? (if only anecdotally)
Re: GitHub Copilot Workspace: Technical Preview
#207While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV
I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…
Once it writes the code, take that into a new session to fix a bug. Repeat with new sessions. Don’t let it read the buggy code, it will just get worse.
Re: GitHub Copilot Workspace: Technical Preview
#208Earlier quoted context omitted.
> and it would have been faster to just do it. False. Obviously this depends on the work, but an LLM is going to get you 80-90% of the way there. It can get you 100% of the way there, but I wouldn't trust it, and you still need to proof read. In the best of times, it is about as good as a junior engineer. If you approach it like you're pair programming with a junior dev that costs <$20/mo then you're approaching it c…
> Obviously this depends on the work, but an LLM is going to get you 80-90% of the way there. No. No it can't. However amazing they are (and they are unbelievably amazing), they are trained on existing data sets. Anything that doesn't exist on StackOverflow, or is written in a language slightly more "esoteric" than Javascript, and LLMs start vividly hallucinating non-existent libraries, functions, method call and pat…
Yes it can. When the project is yet another javascript CRUD app, 80% isn't brand new, never existed before code, but almost-boilerplate that does exist on StackOverflow, on a heavily trodden path where the LLM will get you 80% of the way there.
Re: GitHub Copilot Workspace: Technical Preview
#209I'm not sure why they insist on code spaces instead of running this inside vs code?
Setups/working looks pretty similar to aider [1] that I've been using and liking, Aider is smaller steps than plandex, but plandex went into some kind of loops a couple of times so I stoped using it for now.
Re: GitHub Copilot Workspace: Technical Preview
#210Earlier quoted context omitted.
> And I suspect that kind of story will continue quite a bit as the tech matures. Don't you think that this tech can only get better? And that there will come a time in the very near future when the programming capabilities of AI improve substantially over what they are now? After all, AI writing 300 line programs was unheard of a mere 2 years ago. This is what I think GP is ignoring. Spreadsheets couldn't to do ever…
AI can’t write its own prompts. 10k people using the same prompt who actually need 5000 different things. No improvements to AI will let it read vague speakers’ minds. No improvement to AI will let it get answers it needs if people don’t know how to answer the necessary questions. Information has to come from somewhere to differentiate 1 prompt into 5000 different responses. If it’s not coming from the people using t…
> AI can’t write its own prompts.
What makes you say that? One model can write the prompts of another, and we have seen approaches combining multiple models, and models that can evaluate the result of a prompt and retry with a different one.
> No improvements to AI will let it read vague speakers’ minds. No improvement to AI will let it get answers it needs if people don’t know how to answer the necessary questions.
No, but it can certainly produce output until the human decides it's acceptable. Humans don't need to give precise guidance, or answer technical questions. They just need to judge the output.
I do agree that humans currently still need to be in the loop as a primary data source, and validators of the output. But there's no theoretical reason AI, or a combination of AIs, couldn't do this in the future. Especially once we move from text as the primary I/O mechanism.