Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Workspace: Technical Preview

github.blog

Post reply on HN
Print view View original

281–290 of 333 posts

Re: GitHub Copilot Workspace: Technical Preview

#281

Post by 015a »

throwaway4aday wrote: post #233
015a wrote: post #145

Earlier quoted context omitted.

And GPT-4 is nowhere close to the human brain in terms of coding capabilities, and model advancements appear to be hitting an asymptote. So...

I don't see a flattening. I see a lot of other groups catching up to OpenAI and some even slightly surpassing them like Claude 3 Opus. I'm very interested in how Llama 3 400B turns out but my conservative prediction (backed by Meta's early evaluations) is that it will be at least as good as GPT 4. It's been a little over a year since GPT 4 was released to the public and in that time Meta and Anthropic seem to have ca…

I shouldn't have used the word asymptote; I should have said logarithmic. I don't doubt a best-case situation where we get a GPT-5, GPT-6, GPT-7, etc; each is more capable than the last; just that there will be more months between each, it'll be more expensive to train each, and the gain of function between each will be smaller than the previous.

Let me phrase this another way: Llama 3 400B releases and it has GPT-5 level performance. Obviously; we have not seen GPT-5; so we don't have a sense of what that level of performance looks like. It might be that OpenAI simply has a one year lead, but it might also be that all these frontier model developers are stuck in the same capability swamp; and we simply don't have the compute, virgin tokens, economic incentives, algorithms, etc to push through it (yet). So, Meta pulls ahead, but we're talking about feet, not miles.

Re: GitHub Copilot Workspace: Technical Preview

#282

Post by 015a »

candiddevmike wrote: post #150
singluere wrote: post #112

While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

I've literally never seen an LLM respond negatively to being told "hold on that's not right"; they always say "Oh, you're right!" even if you aren't right.

GPT-4 today: "Hey are you sure that's the right package to import?" "Oh, sorry, you're right, its this other package" (hallucinates the most incorrect response only a computer could imagine for ten paragraphs).

I've seen junior engineers lose half a day traveling alongside GPT's madness before an adult is brought in to question an original assumption, or incorrect fork in the road, or whathaveyou.

Re: GitHub Copilot Workspace: Technical Preview

#283

Post by troupo »

fragmede wrote: post #208
troupo wrote: post #99

Earlier quoted context omitted.

> Obviously this depends on the work, but an LLM is going to get you 80-90% of the way there. No. No it can't. However amazing they are (and they are unbelievably amazing), they are trained on existing data sets. Anything that doesn't exist on StackOverflow, or is written in a language slightly more "esoteric" than Javascript, and LLMs start vividly hallucinating non-existent libraries, functions, method call and pat…

Yes it can. When the project is yet another javascript CRUD app, 80% isn't brand new, never existed before code, but almost-boilerplate that does exist on StackOverflow, on a heavily trodden path where the LLM will get you 80% of the way there.

You've literally repeated what I said

Re: GitHub Copilot Workspace: Technical Preview

#284

Post by sussmannbaka »

imiric wrote: post #279
lispisok wrote: post #239

Earlier quoted context omitted.

You make claims of massive improvements but as an end user I have not experienced such. With the amount of fake and cherrypicked demos in the AI space I dont believe anything until I experience it myself. >Ironically, Waymo's self-driving taxis were launched in several cities in 2023. Does this count? No because usage is limited to a tiny fraction of drive-able space. More cherrypicking.

Just because you haven't used text generation with practically unlimited context windows, insight extraction from personal data, massively improved text-to-image, image-to-image and video generation tools, and ridden in an autonomous vehicle, doesn't mean that the field has stagnated. You're purposefully ignoring progress, and gating it behind some arbitrary ideals. That doesn't make your claims true.

No. The progress is not being ignored. Normal people just have a hard time getting excited for something that is not useful yet. What you are doing here is the equivalent of popular science articles about exciting new battery tech - as long as it doesn’t improve my battery life, I don’t care. I will care once it hits the shelves and is useful to me, I do not care about your list of acronyms.

Re: GitHub Copilot Workspace: Technical Preview

#285

Post by gls2ro »

throwaway4aday wrote: post #243
imiric wrote: post #170

Earlier quoted context omitted.

> And I suspect that kind of story will continue quite a bit as the tech matures. Don't you think that this tech can only get better? And that there will come a time in the very near future when the programming capabilities of AI improve substantially over what they are now? After all, AI writing 300 line programs was unheard of a mere 2 years ago. This is what I think GP is ignoring. Spreadsheets couldn't to do ever…

I don't know about the rest of the developers in the world but my dream come true would be a computer that can write all the code for me. I have piles of notebooks and files absolutely stuffed with ideas I'd like to try out but being a single, measly human programmer I can only work on one at a time and it takes a long time to see each through. If I could get a prototype in 30 seconds that I could play with and then…

If that will be the case then in a finite and small amount of time all your ideas will already have a wide range of implementations/variations because everybody will do the same as you.

It is like now LLMs are on the way to take over (or destroy) content on the web and will take over posts on social media thus making anyone create anything so fast that the incentive to put manual labor into a piece of content is becoming irrelevant in some ways. You work days to write a blog post and publish it and in the same time 1000s of blog posts are published along with yours fighting for the attention of the same audience. who might just stop reading completely because of so much similar things.

Re: GitHub Copilot Workspace: Technical Preview

#286

Post by imiric »

sussmannbaka wrote: post #284
imiric wrote: post #279

Earlier quoted context omitted.

Just because you haven't used text generation with practically unlimited context windows, insight extraction from personal data, massively improved text-to-image, image-to-image and video generation tools, and ridden in an autonomous vehicle, doesn't mean that the field has stagnated. You're purposefully ignoring progress, and gating it behind some arbitrary ideals. That doesn't make your claims true.

No. The progress is not being ignored. Normal people just have a hard time getting excited for something that is not useful yet. What you are doing here is the equivalent of popular science articles about exciting new battery tech - as long as it doesn’t improve my battery life, I don’t care. I will care once it hits the shelves and is useful to me, I do not care about your list of acronyms.

I was arguing against the claim that progress has flatlined, and when I gave concrete examples of recent developments that millions of people are using today, you've now shifted the goalpost to "normal" people being excited about it.

But sure, please tell me more about how AI is a fad.

Re: GitHub Copilot Workspace: Technical Preview

#287

Post by Arainach »

halfmatthalfcat wrote: post #270
ajbt200128 wrote: post #220

Earlier quoted context omitted.

> Shamir Secret Sharing > ChatGPT please don't roll your own crypto, and PLEASE don't roll your own crypto from a LLM. They're useful for other kinds of programs, but crypto libraries need to be to spec, and heavily used and reviewed to not be actively harmful. Not sure ChatGPT can write constant time code :)

People always say this but how else are you going to learn? I doubt many of us who are "rolling our own crypto" are actually deploying it into mission critical contexts anyway.

Asking an LLM to do something for you doesn't involve any learning at all.

Re: GitHub Copilot Workspace: Technical Preview

#288

Post by KeplerBoy »

esperent wrote: post #269
candiddevmike wrote: post #150

Earlier quoted context omitted.

I had ChatGPT output an algorithm implementation in Go (Shamir Secret Sharing) that I didn't want to figure out. It kinda worked, but everytime I pointed out a problem with the code it seemed more bugs were added (and I ended up hating the "Good catch!" text responses...) Eventually, figuring out why it didn't work made me have to read the algorithm spec and basically write the code from scratch, throwing away all of…

The bits GPT4 always gets wrong - and as you say, more and more wrong the further I try to work with it to fix the mistakes - are exactly the bits I want it to do for me. Tedious nested loops that I need to calculate on paper in particular. What it's good for is high level overview and structuring of simple apps, which saves me a lot of googling, reviewing prior work, and some initial typing. After my last attempts t…

Copilot is a decent autocomplete saving you half a line here and there, that's about it.

Re: GitHub Copilot Workspace: Technical Preview

#289

Post by PoignardAzur »

bloomfieldj wrote: post #247
PoignardAzur wrote: post #51

I think Microsoft is going the wrong direction with Copilot (though it's a reasonable direction given their incentives). Right now Copilot is terrible at large changes to complex codebases; the larger and more complex, the worse. But it's great at suggesting very short snippets that guess exactly what you were in the middle of writing and write it for you. I wish Copilot focused more on the user experience at the sma…

I find Cursor’s Copilot++ is miles ahead of GitHub’s in terms of speed and autocomplete helpfulness. They’re also working on the “your cursor should jump to this line next" feature, but I haven’t relied on many of its suggestions yet. It’s available in their vscode fork, but doesn’t seem to be in their docs yet.

Had a glance. Copilot++ looks intriguing, though the landing page is terrible (I can't even see what the demos are trying to show). I might give it a try.

Re: GitHub Copilot Workspace: Technical Preview

#290

Post by rjindael »

idan wrote: post #77

Hello! GitHub Next here, happy to answer questions and unpack how we think about AI tools for developers (spoiler: it's less about codegen and more about helping with the rest of the dev cycle — building an understanding of how the system works, clearly specifying how it should change, etc)

If it is more about code planning, how much different is it than simply telling ChatGPT the overall structure of your code and asking it to give you a rudimentary plan on what to do next? Would it be able to actually execute steps of the plan by generating code, then creating PRs for it? I feel like this is a great tool for our team since my understanding of the announcement is that it is more or less functionally equivalent to hiring another programmer on your team (or, if not that, at the very least having a really useful assistant.) Kudos to the GitHub team and I have immensely enjoyed using Copilot thus far to increase my productivity :-)
Post reply on HN