Just one thing I hate about Copilot is how it needs to be updated two times a day.
GitHub Copilot loses an average of $20 per user per month
211–214 of 214 posts
Re: GitHub Copilot loses an average of $20 per user per month
#212The problem is that the industry has't figured out how to properly pricing LLM applications yet.
Before LLM, it's perfectly possible to spin up a SaaS on 5$ Digital Ocean VM and charge $4.99 per seat monthly. If you're using low overhead techs like Go and SQLite you might get away pretty far with a decent user base.
But LLM is inheirently costly compared to those traditional apps. No matter if you're calling OpenAI or DIY your own GPU cluster it's gonna be way more expensive. Spin your own GPU might ended to be more expensive because utilization problems and upfront costs.
The subscription model was kind of the silver-bullet for SaaS but it's probably not going to work well in the AI era.
OpenAI, Elevenlabs, Runway, and Midjourney: they have subscription model but the quota is strict and tight. The "unlimited" plan is simply pay-as-you-go.
Early wave of LLM products with unlimited subscription models like Github Copilot and Notion AI are probably pricing way too low. $7 or $10 is way too low to support heavy usage.
But charging $50 might scare most user away because it exceeded people's expectation for SaaS. And probably still end up losing money. And hobby users may ended up paying too much for the core users - that will lead us back to sophisticated pricing tiers like Elevenlabs and Runway.
Are there alternatives? I dunno. Maybe implement bring-your-own-key properly? Like OAuth but for LLMs? It's definitely interesting to see how things will turn out eventually.
Re: GitHub Copilot loses an average of $20 per user per month
#213Earlier quoted context omitted.
>That's a problem with all LLMs. They will average what they are trained on, without any reflection what is good and what is not. They shouldn't be called intelligent (AI), they're information meatgrinders. This is an issue with your prompting not the models. If you tell someone "Do this thing" they'll just do it, LLM's too. And how it is done will probably be terrible. If you ask someone "come up with 5 ways to acco…
Fair enough, let's put aside that in Copilot, prompting is automatic. Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory. Compare th…
>Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory.
And?
As a Sr. Engineer, this is exactly how all the slop my Jr's and Mids send me looks. Copy pasted, Stack Overflow, and when I read through Jr code my mind is boggled. I want to shake them "did you even read your own code. do you even know what your code is doing? Why did you do this?" Forget style, best practices, etc,
In my opinion, LLM's produce code that is as good or better than most Jr engineers and as a Sr it is my responsibility to audit, review and test all code. As a Sr level engineer I spend 90% of my time judging/fixing/improving others code, and less than 10% of my time writing my own.
LLM is just another source, and unlike the Jr, I can quickly ask it why it did what it did or to refine it. You ask the Jr to iterate on the project and you won't hear from them until they mention a blocker at tomorrow's stand-up (or you just pair it out and spend 2 hours teaching them, while the LLM turns it around in 15 seconds. There is value in teaching of course, but we build quickly, too).
Re: GitHub Copilot loses an average of $20 per user per month
#214AI is such a huge expectations dichotomy. For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. We start raving about them, so other people take a look expecting that modern LLM's are the greatest thing since sliced bread. They're not quite that, so we get HN comments complaining that AI sucks.
Eh I think people's expectations for things they don't understand are always overblown.
One thing I always find funny is the general expectation that machine learning models are both incredibly generalised and designed based on the way biological systems work, but should also be 100% perfect and never be wrong just like a machine and NOT like a biological system, those things are mutually exclusive; even the best, smartest most physically capable humans will still sometimes spill their coffee, yet we expect coffee-bot 2024 not to do this.
Certainly machines can be much better at a task than humans are, but if that tasks requires generalisation then it's still gonna fuck up from time to time.
Machine learning programs may think a dog is actually a cat sometimes, but afaik they ain't ever called their teacher "Mum" yet.