Live data from Hacker News

Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

news.ycombinator.com

51–60 of 100 posts

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#51
post #28

Earlier quoted context omitted.

Not sure what bedrock is, but last time I looked at their website it claimed it isn't available in my region. I'm in Ireland, so I interpreted that as "Not out yet".

Here is a list of countries Claude is available: https://support.anthropic.com/en/articles/8461763-where-can-...

Lots of countries in that list... but none from the EU :(

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#52
ChatGPT 4 with manual explanations and code snippets seem to work best so far. IDE-integrated tools have the potential, but are really immature yet.

In particular, I haven't found any tools that can do either of the following:

- Comprehend a large codebase. All assistants seem to have a pretty narrow scope for the context, like the current file or few of its neighbors. E.g. they can write some code that already exists somewhere as a ready-to-use utility function, if this function isn't mentioned anywhere near to the location I'm working on.

- Automate anything. IDE integration is extremely minimal in everything I've seen so far. A copilot can spew a new version of the code piece that can be placed with a click, but that's the extent of it. It otherwise can't replace a human in the loop, even if the work is simple, trivial and highly repetitive. E.g. doing a regex-powered "replace all" still cannot be replaced with a natural language query.

- Do any repetitive work without making mistakes or hallucinating something. E.g. a copilot can try to generate an unit test for me and the overall structure can be quite decent, but I cannot ever be sure it won't miss or invent something if it involves creating some complex object and checking all its fields after some transformation.

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#53

In addition to paid subscriptions, you might want to consider running an LLM locally. One of a number of projects enabling this approach is "continue" - https://github.com/continuedev/continue

Haven't found any good local model yet that rivals or come close to gpt-4

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#54

OpenAI API key with GPT4 plus aider[1]: https://github.com/paul-gauthier/aider For reference against the actual product called “Copilot”, I would say this is actually useful, vs Copilot which I would use words like “glorified autocomplete” and “slightly better intellisense” to describe. Only really good for saving you some rote typing periodically. The primary limit with “aider” is the GPT4 context window, so really…

I love aider but it feels like a giant black box that consumes copious credits. The output is amazing though, so I keep using it. But I feel like I have no insight or control. Maybe I should read the docs more carefully.

I use mine in earnest with the full context window filled prompting probably 20-30x a week and have not gotten a monthly bill over $5 USD in a long while. When I first started using it when GPT4 access was limited last year and the tokens were expensive it was a lot more. I think my bills were around $200 USD per month at that point

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#56

For me, personally, Copilot is the best because you are typing and it is just suggesting. Very often I type something, wait 2 seconds for its answer (knowing it is going to get it right 80% of the time) and press tab. Sure, it misses a ton, but I see it as a great autocorrect that is always there trying to help. Now, copilot chat and everything else are very bad. I don't like them. If you are "intentional" I usually…

Have you ran into issues with Copilot where it does things like:

- Doesn't close brackets / adds too many closing brackets

- Spams completions while you're writing comments

- Doesn't auto import the required libraries after accepting a suggestion

- No multi cursor mode

- Refuses to name variables

- Refuses to trigger in the middle of a line

- Lower quality code than ChatGPT (probably because they use a smaller model)

Asking because I was also a heavy Github Copilot user but after 1-2 years of constant subpar UX and seemingly no effort from the Copilot team to fix it, I just went ahead and built my own extension that fixes all of the above: https://double.bot

Also it wasn't on your list but you mentioned GPT-4 is smart. You should try the new Claude 3 Opus, we implemented it on Monday and have been getting super positive feedback from users!

Re: Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

#59

I have been a big user of ChatGPT-4 ever since it came out. I have used in various forms. Through the chatbot, through GitHub Copilot etc. For the past year, I have looked high and low to see an alternative that performed better. Nothing came close. Until a few days ago with Anthropic released Claude 3. For software development work, it is very very impressive. I've been running GPT-4 and Claude 3 Opus side by side f…

How do you get access to Claude 3? It doesn’t seem to be available yet.

Wasn't aware that it's not widely available but if you can't find it anywhere else, we have it available in our free tier on Double: https://double.bot
Post reply on HN