21–30 of 225 posts
It's good at writing new code, with sufficient prompting. But the big open question as of now for engineering orgs is - can it edit existing code like developers, just by instructions. Is there any hands on experience anyone has on copilot-x?
For best results, pasting any relevant documentation into the prompt can help.
What value does a developer deliver when their entire process is done by an LLM? Is there no desire for a creative process?
The dev gave the initial idea to the LLM. That's the creative process. Everything after that, arguably, is just technical details in order to realize the idea. Sure, implementation requires plenty of creativity, but of different kind.
I've used LLM to generate a lot of code recently on side projects. It's a 10x jump in productivity, but it can only reliably do 50-80% of the work, and the last tail needs editing, verification, setup with infrastructure, etc.
It won't read your mind, you need to iterate, re-create, and guide. And each of those 3 relies on a working knowledge of software, libraries, tech, and user experience to get right.
For now.
Earlier quoted context omitted.
This is not even wrong. The level of analogies people use in the first few months of $hype_cycle is mind boggling.
It's a spot on - moving up a level in the abstraction hierarchy.
A: Certainly, as an AI language model I'm happy to oblige. Here is the algorithm.
Human: I used a higher abstraction level! I have solved P == NP!
Would it be possible to significantly improve an LLM using such unsupervised sessions?
We should have never listened to Richard Stallman. /s
Is anyone letting an LLM code and run its code by itself, then iteratively fix any bugs in it without human intervention until it e.g. passes some black box tests? Would it be possible to significantly improve an LLM using such unsupervised sessions?
Earlier quoted context omitted.
It's a spot on - moving up a level in the abstraction hierarchy.
Human: ChatGPT, please devise an algorithm that solves the traveling salesman problem in polynomial time. A: Certainly, as an AI language model I'm happy to oblige. Here is the algorithm. Human: I used a higher abstraction level! I have solved P == NP!
And you still need to have some model of the code structure, you need to understand what is going on to go back and forth with it. It takes care of a part of the work that is quite standard so you work in a different level, the analogy is with a compiler taking care of certain optimisation part so that you do not have to invent it every time. If you think that you can build anything meaningful with chatGPT with one line prompts I would suggest you try to engage yourself in the context of a language/paradigm you are only a novice with to see how it works and the parts that can be a good learning experience, engaging and entertaining. Do not use some obscure language and libraries because it will start hallucinating a lot though.
You don’t mention it explicitly, but I assume you’ve manually copied and pasted all the code, as well as the various patches with updates? In my experience, that quickly makes new suggestions from ChatGPT go out of sync with the actual state of the code. Did you occasionally start the conversation over and pasted in all the code you currently had, or did this not turn out to be an issue for you?
A good example looks like:
``` I am trying to model associative memory that i may attach to a gpt model
here is the code for the memory:
....
can we keep the input vectors in another array so we can fetch items from it directly instead of having to reconstruct? ```
Is anyone letting an LLM code and run its code by itself, then iteratively fix any bugs in it without human intervention until it e.g. passes some black box tests? Would it be possible to significantly improve an LLM using such unsupervised sessions?
Also, gpt-4 does this much better than gpt-3.5, but gpt-4 is really slow, so the iteration process can take tens of minutes.
Earlier quoted context omitted.
The dev gave the initial idea to the LLM. That's the creative process. Everything after that, arguably, is just technical details in order to realize the idea. Sure, implementation requires plenty of creativity, but of different kind.
Believe me, only a dev can get this working. Maybe in the future, LLM wizards will conjure all our technology, but at this point, having a working knowledge of all APIs from 2021 is an assistive technology, not a magical code-machine. I've used LLM to generate a lot of code recently on side projects. It's a 10x jump in productivity, but it can only reliably do 50-80% of the work, and the last tail needs editing, veri…