Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Workspace: Technical Preview

github.blog

Post reply on HN
Print view View original

191–200 of 333 posts

Re: GitHub Copilot Workspace: Technical Preview

#191

Post by chasd00 »

mucle6 wrote: post #177

I wonder if we can use "copilot-ability" as a proxy for code base complexity. In other words, if copilot can't help you write code, then could that mean your code base is too complicated for a beginner to understand and modify?

I hope not but i bet you're right. I'm sure there's lots of people working on llms to generate a "cognitive complexity" score like some of the static code analyzers we have. It will be so ripe with false positives as to be effectively worthless however the score is an easy thing to build a metric, report, and policy on... just like with the analyzers today.

Re: GitHub Copilot Workspace: Technical Preview

#192

Post by darepublic »

studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

Ever been frustrated with a piece of software? Or wished for some software to exist to solve a problem? Well just point these nifty tools at it and watch the solutions magically materialize. That will comfort you somewhat after being bummed out over obsoletion. But if you find you can't materialize the desired software quick time then.. I guess at least for now humanity still required

Re: GitHub Copilot Workspace: Technical Preview

#193

Post by layer8 »

mucle6 wrote: post #177

I wonder if we can use "copilot-ability" as a proxy for code base complexity. In other words, if copilot can't help you write code, then could that mean your code base is too complicated for a beginner to understand and modify?

That assumes that it would be reasonable to expect a beginner to understand the code base of any nontrivial application or system. I don’t think that’s a reasonable expectation, if only because there are countless requirements, assumptions, experiences made, priorities and design decision that have to be known to understand the code.

It’s also a question of economy. You can always try to make a code base easier to understand and to document anything and everything, but it comes at a cost. It doesn’t help if a code base is in principle understandable by a beginner if they have to spend months reading documentation first, and someone had to spend months if not years to write it.

Re: GitHub Copilot Workspace: Technical Preview

#194

Post by lispisok »

imiric wrote: post #170
lelandbatey wrote: post #151

Earlier quoted context omitted.

I've seen it already. A small business owner (one man show) friend of mine with zero developer experience was able to solve his problem (very custom business specific data -> calendar management) in a rough way using ChatGPT. But it got past about 300 lines long and really started to get bad. He'd put dozens of hours of time on his weekend to getting it to where it was by using ChatGPT over and over, but eventually i…

> And I suspect that kind of story will continue quite a bit as the tech matures. Don't you think that this tech can only get better? And that there will come a time in the very near future when the programming capabilities of AI improve substantially over what they are now? After all, AI writing 300 line programs was unheard of a mere 2 years ago. This is what I think GP is ignoring. Spreadsheets couldn't to do ever…

You are extrapolating from when we saw huge improvements 1-2 years ago. Performance improvements have flatlined. Current AI predictions reminds me of self-driving car hype from the mid 2010s

Re: GitHub Copilot Workspace: Technical Preview

#195

Post by simonw »

studenthrow3831 wrote: post #3

Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme prod…

These tools make people who know software engineering massively more productive.

Given the choice between an LLM-assisted non-engineer and an LLM-assisted experienced software engineer, I know who I would want to work with - even if the non-engineer was significantly cheaper.

Re: GitHub Copilot Workspace: Technical Preview

#196

Post by chasd00 »

Bjorkbat wrote: post #166
nojvek wrote: post #158

I use Github co-pilot but 55% productive is a bullshit number. Perhaps 1% may be. Most of co-pilot suggestions are either simple pattern matches or subtle hallucinations where I have to catch and fix silly bugs. Github Chat is not very useful at understanding what the code is doing. Tried it once or twice and gave up. The hype will help with Microsoft Stock though. Seems like bean counter management is taking over.

I was actually about to make a comment on that. They got that number from some dumb "experiment" they did where told two groups of developers to implement an HTTP server in Javascript. The group with Copilot got the job done 55% faster ( https://github.blog/2022-09-07-research-quantifying-github-c... ) So yeah, they did an experiment with <100 developers asking them to create implement something that only took the co…

That's really disappointing and a little bit scary. That seems like a task tailor fit for an LLM, it's been done a million times before, follows an established protocol, and the implementations are all basically the same. Granted a lot of code is like that but a more realistic task may have been something like "create an online ordering system for a cake company that sells different kinds of cakes, has a sale every so often, and uses coupons printed in the local paper". That task is more ambiguous and better reflects what software devs are tasked with day to day IMO.

Re: GitHub Copilot Workspace: Technical Preview

#197

Post by theshrike79 »

packetlost wrote: post #26
cedws wrote: post #22

Earlier quoted context omitted.

Less time spent writing code is more time you can spend thinking about those hard parts, no?

Yes? But it's commonly understood that reading code is harder than writing code. So why force yourself into a reading-mostly position when you don't have to? You're more likely to get it wrong. There are other ways to decrease typing time.

It's not harder unless you write hard to read code.

> “Indeed, the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as part of the effort to write new code. ...[Therefore,] making it easy to read makes it easier to write.” - Robert C. Martin in Clean Code

LLMs make exceptionally clean code in my opinion. They don't try to be fancy or "elegant", they just spit out basic statements that sometimes (or most of the time) do what you need.

Then you _read_ what it suggests, with a skilled eye you can pretty much glance and see if it looks good and test it.

Re: GitHub Copilot Workspace: Technical Preview

#198

Post by krainboltgreene »

Vuizur wrote: post #163
krainboltgreene wrote: post #15

Earlier quoted context omitted.

> Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools Because we've seen similar hype before and we know what impactful change looks like, even if we don't like the impact (See: Kubernetes, React, MongoDB). > executing large-scale changes in entire repositories in 3 years Is this actually happening? I haven't seen any evidence of that.

>executing large-scale changes in entire repositories in 3 years You can look at SWE-Agent, it solved 12 percent of the GitHub issues of their test dataset. It probably depends on your definition of large-scale. This will get much better, it is a new problem with lots of unexplored details, and we will likely get GPT-5 this year, which is supposed to be a similar jump in performance as from 3.5 to 4 according to Altm…

This is a laughable definition of large-scale. It's also a misrepresentation of that situation: It was 12% of issues in a dataset for the top 5000 repositories pypy packages. Further "solves" is a incredibly generous definition, so I'm assuming you didn't read the source or any of the attempts to use this service. Here's one where it deletes half the code and replaces network handling with a comment to handle network handling: https://github.com/TBD54566975/tbdex-example-android/pull/14...

"this will get much better" is the statement I've been hearing for the past year and a half. I heard it 2 years ago about the metaverse. I heard it 3 years ago about DAOs. I heard it 5 years about block chains...

What I do see is a lot more lies. Turns out things are zooming along at the speed of light if you only read headlines from sponsored posts.

Re: GitHub Copilot Workspace: Technical Preview

#199

Post by Zambyte »

BoorishBears wrote: post #122
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

Creating this using Open Source LLMs would be like saying you tried A5 Wagyu by going to Burger King, respectfully. I think benchmarks are severely overselling what open source models are capable of compared to closed source models.

I really don't think they're being over sold that much. I'm running llama 3 8b on my machine, and it feels a lot like running claude 3 haiku with a much lower context window. Quality wise it is surprisingly nice.

Re: GitHub Copilot Workspace: Technical Preview

#200

Post by AlexCoventry »

bastardoperator wrote: post #189
candiddevmike wrote: post #150

Earlier quoted context omitted.

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

I just asked it the same question and this was the answer it gave me: go get go.dedis.ch/kyber/v3 LOL...

That's pointing to a fairly solid implementation, though (I've used it.) I would trust it way before I'd trust a de novo implementation from ChatGPT. The idea of people using cryptographic implementations written by current AI services is a bit terrifying.
Post reply on HN