Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot loses an average of $20 per user per month

thurrott.com

Post reply on HN
Print view View original

181–190 of 214 posts

Re: GitHub Copilot loses an average of $20 per user per month

#181

Post by nvm0n2 »

YeGoblynQueenne wrote: post #179
bryanlarsen wrote: post #49

AI is such a huge expectations dichotomy. For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. We start raving about them, so other people take a look expecting that modern LLM's are the greatest thing since sliced bread. They're not quite that, so we get HN comments complaining that AI sucks.

>> For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. Aren't you overgeneralising a bit? Not even I would say that CNNs for image classification, or Deep-RL for board game-playing are a "continual disappointment" and they certainly predate LLMs. Are you talking about NLP? Even Neural Turing Machines were quite capable in language pairs…

Pre-2022 most AI was disappointing not because it was unimpressive but because it near exclusively took the form of blog posts or university papers that announced something interesting but unusable. Either because it was merely an academic curiosity (AlphaGo) or because the tech wasn't being released for public usage at all, and wasn't really easy to replicate externally either, or because there was no obvious way to apply it to normal problems. So we all got used to this constant year-in-year-out stream of "amazing" "breakthroughs" that ended up being a thats-cool followed by a shrug.

RLHF trained GPT3 and then DALLE/Midjourney/Stable Diffusion changed all that. Suddenly AI not only got good, but the field broke loose of the inane and insane obsession with pseudo-safety that had been holding it back. Now the rest of us can use it without dropping $5M on a GPU cluster and hiring a dozen researchers first. AI is no longer a disappointment.

Re: GitHub Copilot loses an average of $20 per user per month

#182

Post by nvm0n2 »

JohnBooty wrote: post #155
marcosdumay wrote: post #95

Earlier quoted context omitted.

> for a lot of developers (but not all), the value Github Copilot is incredibly high, much more than $20/month. I really doubt this. IMO, the jury is still out deciding if the value is above zero for enough people to matter.

Are we talking about Copilot in particular or AI code assistants in general? The value, for me, is extremely high. My teammates feel the same. Our shared opinion/experience is that ChatGPT 4 is better than Copilot in general but Copilot shines in-editor because it's aware of your project. So we use both in tandem. They mostly use Chat GPT and I split about 50%/50%. (Note: I'm using the Copilot X beta which I believe…

https://aider.chat/ <-- secret weapon, use GPT4 for coding via the API with much better support than what Copilot gives you.

Re: GitHub Copilot loses an average of $20 per user per month

#183

Post by nvm0n2 »

outside1234 wrote: post #151
treis wrote: post #149

Earlier quoted context omitted.

They just need to keep it going until GPUs get cheaper. The point is to build/protect the monopoly until that happens. Then costs come down and they start printing an absurd amount of money.

The problem is that they don't have a monopoly. Not hard to see how CodeLlama and other future open source models can't stand-in here for GPT-4 at dramatically lower cost.

Because:

1. GPT4 is on a different scale to CodeLlama

and

2. If open models do catch up, they will have moved on to GPT5 or GPT6 and the gap will have widened again

Re: GitHub Copilot loses an average of $20 per user per month

#184

Post by nvm0n2 »

wrs wrote: post #126
_fat_santa wrote: post #91

Earlier quoted context omitted.

I tried Copilot in the early days but stopped using it pretty quickly as it just wasn't that useful in an already established codebase. Often times I'm not writing completely new code but rather implementing a pattern that I've previously defined somewhere else. In this case Copilot just completely falls over because it doesn't know about that existing code. My other gripe is language specific but in VSCode it breaks…

I have found it surprisingly good at some pretty sophisticated “grunt work”. For example, I’m converting some code to TypeScript. Looking at a long and twisty “getSomething” JavaScript function that builds up an object one field at a time, it can complete “interface Something {“ with all those fields, with types! It might duplicate or miss a field sometimes but it makes fewer mistakes than I would. I’m pretty sure it…

It uses the contents of other editor tabs.

Re: GitHub Copilot loses an average of $20 per user per month

#185

Post by WillPostForFood »

SadWebDeveloper wrote: post #167
WillPostForFood wrote: post #138

Earlier quoted context omitted.

$20 a month is incredibly expensive? If it saves 15 minutes over a month, it pays for itself (assuming your work is ~$100/hr). If it can provide any positive value, it is going to be worth more the $5 a month.

Maybe if you live around the bay area where you get paid absurd money by the hour for a job that can be done overseas for 10 bucks, outside or the rest of the world that 100 usd is per week tops, thats way too much i would pay for service that only barely helps my in my day to day... If you need to constantly use chatgpt/copilot to do your job properly, maybe its time to learn something else because your job will be…

Does it barely help you, or can it completely automate your job? You can't have it both ways.

I find it hard to believe there is a scenario where it actually helps you, but only provides $5 of value in a month.

What exact kind help is that? If it is able to provide $5 in help, you could likely scale that to $20.

Re: GitHub Copilot loses an average of $20 per user per month

#186

Post by bhouston »

norir wrote: post #124
simonw wrote: post #98

Earlier quoted context omitted.

From that article: 1.5m users and up to 50% of their code is now being produced by Copilot. Sounds to me like the verdict on its value is in already.

I mean you could have made a similar argument about the productivity benefits of smoking cigarettes 100 years ago. Just because a lot of people are doing something doesn't mean it's valuable or that we have an accurate picture of the cost/benefits. The verdict is still very much out on LLMs.

> The verdict is still very much out on LLMs.

Any new product that gained +1M paying subscribers (Github Copilot) in its first year is a success. You cannot like it, that is allowed. And it may not help you, but +1M subscribers is a lot of people.

They are definitely here to stay until they are superseded by even better technology or it gets sued into oblivion.

Re: GitHub Copilot loses an average of $20 per user per month

#188

Post by IshKebab »

baz00 wrote: post #154
bryanlarsen wrote: post #49

AI is such a huge expectations dichotomy. For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. We start raving about them, so other people take a look expecting that modern LLM's are the greatest thing since sliced bread. They're not quite that, so we get HN comments complaining that AI sucks.

They do suck unconditionally. They're like having a mediocre person working with you that you have to verify and validate all the time that sucks up a ton of money and electricity and generally does bad things to the environment. Edit: I haven't even started on the social and political impact this has either.

You missed the point. And you're also completely wrong. Copilot is extremely useful and in general a huge timesaver.

Yes if you tell it to write an entire program it will get it wrong and you'll spend some time verifying things. But that's not a sane way to use it. As a very clever auto-complete it's fantastic. It's also pretty great at getting past "blank page syndrome". Even if what it spits out is wrong it's still helpful to get you started.

Re: GitHub Copilot loses an average of $20 per user per month

#189

Post by gwern »

esalman wrote: post #92
hesdeadjim wrote: post #5

The moment I can train Copilot on my codebase and improve its predictions, I’d gladly pay more than $20. Even now, I find that I can’t live without it simply as a slightly more intelligent auto-complete.

It's amazing and all, but can be so subtly wrong. I let it autocomplete a function for generating time zone offset. It sneaked a minus sign in there so central US offset would be +0600 instead of -0600. My manager caught it during code review. We both are proponents of copilot but it was still embarrassing for me.

Sounds like something a few tests would have caught. I've found my GPT-4 coding use to be much more satisfactory when I have it generate a few dozen test cases after finishing (either my code or its code), and then auto-repair based on any errors.

Re: GitHub Copilot loses an average of $20 per user per month

#190

Post by baz00 »

IshKebab wrote: post #188
baz00 wrote: post #154

Earlier quoted context omitted.

They do suck unconditionally. They're like having a mediocre person working with you that you have to verify and validate all the time that sucks up a ton of money and electricity and generally does bad things to the environment. Edit: I haven't even started on the social and political impact this has either.

You missed the point. And you're also completely wrong. Copilot is extremely useful and in general a huge timesaver. Yes if you tell it to write an entire program it will get it wrong and you'll spend some time verifying things. But that's not a sane way to use it. As a very clever auto-complete it's fantastic. It's also pretty great at getting past "blank page syndrome". Even if what it spits out is wrong it's still…

No I didn't miss the point, not even slightly. You didn't read mine and are blinded by the insane virtues.

Do you really think it's fine blowing 400 watts because you can't be arsed to think or do not have the creative intelligence to get over the blank page syndrome and have to lean on a crutch?

Post reply on HN