Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tips for better coding with ChatGPT

nature.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: Tips for better coding with ChatGPT

#81

Post by spaceman_2020 »

agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

I haven’t been able to get GPT-4 API access months after applying and being a paid subscriber.

The world right now seems to be divided more into “have GPT-4 API/ don’t have GPT-4 API”.

Re: Tips for better coding with ChatGPT

#82

Post by elemos »

spaceman_2020 wrote: post #81
agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

I haven’t been able to get GPT-4 API access months after applying and being a paid subscriber. The world right now seems to be divided more into “have GPT-4 API/ don’t have GPT-4 API”.

Same here. Instead, I get to pay for ChatGPT and have people tell me it’s inferior.

Re: Tips for better coding with ChatGPT

#83

Post by fnordpiglet »

Jeff_Brown wrote: post #80
agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

> now infinitely more productive How much more productive do you feel you are coding with an LLM? As another HN user said, to me it's like a talking dog -- incredible yet useless.

I feel it gives me about a 30% lift on mechanical tasks and a 60% lift on learning / unblocking in areas of ambiguity. I use GPT for the mechanical tasks and ChatGPT for the learning tasks. An example of the learning would be “explain to me the use of Box::pin in the context of rust futures” or some such, but also sometimes some common idiom I’m brain farting on. Searching Kagi will yield the answers, just more slowly and deeply embedded in some document or stackoverflow answer vomit requiring lot of wasted effort that fully distracts me from my flow. The fact I can ask follow up questions on areas of ambiguity is useful. When it hallucinates it generally means I’m in an area that’s either undefined as of yet, or is really niche. The nice thing about programming is hallucination feedback is basically instant so I then pull out Kagi and research a bit, and maybe 90% of the time it’s just not possible.

There has been some work done on generating code in a feedback cycle to winnow out hallucinations and it seems to work fairly well [1]- I think 99% of the challenges LLM face are primarily related to a lack of constraint, optimization, agency, and solver feedback. As they get integrated into a system with the ability to inform and constrain and guide using classic AI techniques their true value will be attainable. But they’re pretty useful even today.

N.b., I’m a 32 year veteran distinguished engineer level at FAANG and adjacent firms that programs daily.

1 https://voyager.minedojo.org

Re: Tips for better coding with ChatGPT

#84

Post by pixl97 »

Swizec wrote: post #60
endisneigh wrote: post #31

I've used GPT-4 before it changed, and though impressive, writing code has never been the bottleneck for me personally. When these things can understand the business requirements and tell me what I should be building and why, with detailed sensible reasoning then I'll be hyped.

> When these things can understand the business requirements I’ve been looking into this. Nothing definitive yet, but my hunch is that current LLM’s struggle because they lack curiosity. They will answer your question, however vague, with the first most obvious answer they can think of. This is great, if you’re a junior team member. Super talented, very eager. But a more senior engineer approaches the problem differe…

>because they lack curiosity.

This is because curiosity requires free energy, hence it is very expensive when you're limited on very expensive compute.

This is what tree of thought is attempting to simulate in some ways. Build a set of multiple questions around the original question and then build on and prune that list based on a 'show your work' set of steps, and then keep iterating.

Humans naturally solve the halting problem when thinking about things... we work on something long enough without a break and we'll pass out. Maybe when we wake, eat, and go to work we'll stop working on the same problem. But an LLM never sleeps. In theory with TOT and no time limit, you could find out your AutoGPT spent 10 million in computing resources contemplating navel lint. So, there are a number of unsolved problems there.

What really becomes concerning is if Nvidia achieves its goals of speeding up training/inference by 1 million times in the next few years, and if the amount of compute we produce increases by a few million times. You and me simply can't use hundreds of minds thinking for years straight and machines could.

Re: Tips for better coding with ChatGPT

#85

Post by agentcoops »

Swizec wrote: post #79
agentcoops wrote: post #74

Earlier quoted context omitted.

I think you’re still anthropomorphizing LLMs too much by saying the problem is a lack of curiosity. It’s an engineering problem: you haven’t figured out the correct “context” from which business requirements would follow. (And why Microsoft is so incredibly well positioned for the future)

Curiosity is how you build that context.

I'm sure in your career you've seen countless attempts by 1000+ employee firms to spinup an ops team that must construct a giant spreadsheet of product priorities between customers, PMs and engineering teams -- versus having all of Salesforce, Slack, emails, meeting transcripts, user support messages, vectorised and fed into an LLM that any employee can converse with. The latter is not a trivial problem by any means and many, many implementation details, but I'm sure it's such an exciting time to be at Microsoft. Stratechery had a fantastic article about this.

If I had to make a bet, I suspect Google and Meta will royally lose this generation: Apple will dominate personal access to LLMs, Microsoft will dominate business access to LLMs -- and honestly Palantir for the first time terrifies me. The current generation of LLMs are not about automation as such, but helping individuals accomplish self-directed goals (as possible in language and code).

Re: Tips for better coding with ChatGPT

#86

Post by TechBro8615 »

agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

Maybe, but I can't help but notice the "programmatic control" is over someone else's computer. Those who care the most about having programmatic control over their computers are also those who would rather send their prompts to their own GPU rather than an Azure server farm. I believe there will be a programming revolution built on the foundation of LLMs, but it won't really take off until we can use local LLMs for the bulk of the processing.

Re: Tips for better coding with ChatGPT

#87

Post by w0m »

Jeff_Brown wrote: post #80
agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

> now infinitely more productive How much more productive do you feel you are coding with an LLM? As another HN user said, to me it's like a talking dog -- incredible yet useless.

> to me it's like a talking dog

That's if you're explicitly asking it questions. Have you sat down and coded with Copilot offering suggestions as you went? It's honestly incredibly helpful, especially when leveraging a new language or stack.

Re: Tips for better coding with ChatGPT

#88

Post by ducharmdev »

apples_oranges wrote: post #13
chank wrote: post #6

> Trust but verify... Based on my own attempts to have ChatGPT generate code for me, it's better to NOT trust it. It tends to hilucinate even with simple requirements.

I think they should use a different name altogether for chatGPT 4 as it is very different and superior to 3.5. And when people say chatGPT doesn't work for me, I would like to know which one they tried.

I was chatting with someone I know that has a paid subscription for ChatGPT 4. I was venting about the hallucinations I was getting for a certain problem; they mentioned how much better v4 is, and proceeded to plug my prompt into it. The answer it gave was actually a more blatant lie compared to the responses I got from v3.5.

Re: Tips for better coding with ChatGPT

#89

Post by StackOverlord »

Jeff_Brown wrote: post #80
agentcoops wrote: post #21

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads th…

> now infinitely more productive How much more productive do you feel you are coding with an LLM? As another HN user said, to me it's like a talking dog -- incredible yet useless.

I use GPT to write code sometimes instead of importing a library. For instance a function to breadth first traverse a directed acyclic graph and slice it into levels. Another one to find a node in a nested graph using partial paths. I could have written those functions, but GPT4 does it correctly and faster.

Re: Tips for better coding with ChatGPT

#90

Post by ducharmdev »

On sites like Stack Overflow, incorrect and subtley wrong answers are often peppered with follow-up answers. But in a ChatGPT session, I alone have to critique its output, without contextual info regarding the data's source and without a community to help me critique it.

I find that a bit exhausting; beyond simple use-cases, I've found it easier to just do it myself.

Post reply on HN