Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Workspace: Technical Preview

github.blog

Post reply on HN
Print view View original

161–170 of 333 posts

Re: GitHub Copilot Workspace: Technical Preview

#161

Post by SCUSKU »

studenthrow3831 wrote: post #28
packetlost wrote: post #10

Earlier quoted context omitted.

The hardest part of software development is not writing code, full stop. It never has been and it never will be. The hard part is designing, understanding, verifying, and repairing complex systems. LLMs do not do this, even a little bit.

I guess my worst fear is not "no more jobs because AI can code" but "no more junior jobs because AI can code under the supervision of a senior". SWE jobs will exist, but only seniors will have them and juniors are never hired. Maybe the occasional "apprentice" will be brought on, but in nowhere near the same amount. Where my blind hope lies more specifically is in networking into one of those "apprentice" roles, or m…

If you're a junior looking for a job, it's always a tough time. Getting your first gig is insanely brutal (college career fairs help a lot). That said, I wouldn't give up and blame AI for "taking our jerbs". I would say the current macroeconomic conditions with higher interest rates have reduced the amount of developer headcount companies can support. AKA companies are risk averse right now, and juniors are a risk (albeit a relatively low cost).

If I were in your shoes, I would just stop consuming the doom and gloom AI content, and go heads down and learn to build things that others will find useful. Most importantly, you should be having fun. If you do that you'll learn how to learn, have fun, build a portfolio, and generally just be setting yourself up to succeed.

Re: GitHub Copilot Workspace: Technical Preview

#162

Post by daviding »

One side effect that I don't think I like of these tools is the inevitable push for developers to just use the most popular languages. Javascript and python are the main LLM trained languages, so it just self reenforces that it is best to use those for everything. There is something a bit sad about that. I guess the dream is that they are just intermediatory data anyway, like a chatty bytecode layer.

Re: GitHub Copilot Workspace: Technical Preview

#163

Post by Vuizur »

krainboltgreene wrote: post #15
studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

> Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools Because we've seen similar hype before and we know what impactful change looks like, even if we don't like the impact (See: Kubernetes, React, MongoDB). > executing large-scale changes in entire repositories in 3 years Is this actually happening? I haven't seen any evidence of that.

>executing large-scale changes in entire repositories in 3 years

You can look at SWE-Agent, it solved 12 percent of the GitHub issues of their test dataset. It probably depends on your definition of large-scale.

This will get much better, it is a new problem with lots of unexplored details, and we will likely get GPT-5 this year, which is supposed to be a similar jump in performance as from 3.5 to 4 according to Altman.

Re: GitHub Copilot Workspace: Technical Preview

#164

Post by nforgerit »

studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

Don't fret. What people call AI these days is just a gigantic economically unsound bullshit generator (some underlying ideas might be valuable though) that passed very stupid tests. It is brutally marketed like blockchain & crypto by some sociopaths from Silicon Valley, their mini-mes and middle management hell which needs to double down on bad investments.

The bigger problem I see is the economical situation.

Re: GitHub Copilot Workspace: Technical Preview

#165

Post by singluere »

lostintangent wrote: post #125
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

I can definitely echo the challenges of debugging non-trivial LLM apps, and making sure you have the right evals to validate progress. I spent many hours optimizing Copilot Workspace, and there is definitely both an art and a science to it :) That said, I’m optimistic that tool builders can take on a lot of that responsibility, and create abstractions that allow developer to focus solely on their code, and the proble…

For sure! As a user, I would love to be able to have some sort of debugger like behavior for debugging the LLM's output generation. Maybe some ability for the LLM to keep on running some tests until they pass? That sort of stuff would make me want to try this :)

Re: GitHub Copilot Workspace: Technical Preview

#166

Post by Bjorkbat »

nojvek wrote: post #158

I use Github co-pilot but 55% productive is a bullshit number. Perhaps 1% may be. Most of co-pilot suggestions are either simple pattern matches or subtle hallucinations where I have to catch and fix silly bugs. Github Chat is not very useful at understanding what the code is doing. Tried it once or twice and gave up. The hype will help with Microsoft Stock though. Seems like bean counter management is taking over.

I was actually about to make a comment on that. They got that number from some dumb "experiment" they did where told two groups of developers to implement an HTTP server in Javascript. The group with Copilot got the job done 55% faster (https://github.blog/2022-09-07-research-quantifying-github-c...)

So yeah, they did an experiment with <100 developers asking them to create implement something that only took the control group 3 hours to finish from scratch, and from this we got the "55% more productive" statistic.

Re: GitHub Copilot Workspace: Technical Preview

#168

Post by Bjorkbat »

You can see my earlier comment somewhere here, but I feel obliged to remind everyone that the claim that Github Copilot made developers "55% more productive" came from a study where they asked 100 developers to implement an HTTP server in Javascript, split the group up roughly 50/50, and gave one group Github Copilot. The Copilot group did it in an hour and 11 minutes, whereas the control group got it done 2 hours and 41 minutes.

https://github.blog/2022-09-07-research-quantifying-github-c...

That's where the 55% number is coming from. It's coming from this experiment, and only this experiment.

So yeah, if you're wondering why you aren't somehow 50% more productive when using Github Copilot, it's probably because you're not implementing a simple task that's been done to death in countless tutorials.

Re: GitHub Copilot Workspace: Technical Preview

#169

Post by torginus »

candiddevmike wrote: post #150
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

An alterinative to this workflow that I find myself returning to is the good ol' nicking code from stackoverflow or Github.

ChatGPT works really well because the stuff you are looking for is already written somewhere and it solves the needle-in-the-haystack problem of finding it, very well.

But I often find it tends to output code that doesn't work but eerily looks like it should, whereas Github stuff tends to need a bit more wrangling but tends to work.

Re: GitHub Copilot Workspace: Technical Preview

#170

Post by imiric »

lelandbatey wrote: post #151
sixhobbits wrote: post #20

Earlier quoted context omitted.

Accountants thought spreadsheets would kill their profession, instead demand for them exploded. Compilers made it much easier to code compared to writing everything in Assembly. Python made it much easier to code than writing C. Both increased the demand for coders. Code is a liability, not an asset. The fact that less technical people and people who are not trained engineers can now make useful apps by generating mi…

I've seen it already. A small business owner (one man show) friend of mine with zero developer experience was able to solve his problem (very custom business specific data -> calendar management) in a rough way using ChatGPT. But it got past about 300 lines long and really started to get bad. He'd put dozens of hours of time on his weekend to getting it to where it was by using ChatGPT over and over, but eventually i…

> And I suspect that kind of story will continue quite a bit as the tech matures.

Don't you think that this tech can only get better? And that there will come a time in the very near future when the programming capabilities of AI improve substantially over what they are now? After all, AI writing 300 line programs was unheard of a mere 2 years ago.

This is what I think GP is ignoring. Spreadsheets couldn't to do every task an accountant can do, so they augmented their capabilities. Compilers don't have the capability to write code from scratch, and Python doesn't write itself either.

But AI will continually improve, and spread to more areas that software engineers were trained on. At first this will seem empowering, as they will aid us in writing small chunks of code, or code that can be easily generated like tests, which they already do. Then this will expand to writing even more code, improving their accuracy, debugging, refactoring, reasoning, and in general, being a better programming assistant for business owners like your friend than any human would.

The concerning thing is that this isn't happening on timescales of decades, but years and months. Unlike GP, I don't think software engineers will exist as they do today in a decade or two. Everyone will either need to be a machine learning engineer and directly work with training and tweaking the work of AI, and then, once AI can improve itself, it will become self-sufficient, and humans will only program as a hobby. Humans will likely be forbidden from writing mission critical software in health, government and transport industries. Hardware engineers might be safe for a while after that, but not for long either.

Post reply on HN