Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: GPT Repo Loader – load entire code repos into GPT prompts

github.com

Post reply on HN
Print view View original

31–40 of 162 posts

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#31

Post by xwdv »

antibasilisk wrote: post #20
tobiasSoftware wrote: post #14

Earlier quoted context omitted.

I don't see programming going away for this reason. Think about it, if you have to carefully describe what you want to do to an AI - you are just writing a program. Only a program is deterministic and will do what you tell it to, whereas an AI may or may not. The future that I see, coding and AI are divided into two camps. The one is what we would call "script kiddies" today - people who don't understand how to write…

Between GPT-3 and GPT-4 the precision required for prompts was decreased significantly. In theory it should reach the point where a project manager type person would be able to describe what is needed and it would simply do it, the main thing missing from attaining that is that GPT-4 basically never responds to questions with requests for clarification, otherwise, a whole team of developers could be reduced to just o…

That’s not very impressive. With very little precision I can throw a few keywords into google and get a full answer and perhaps some code snippets from stack overflow to solve whatever problem I have in the moment.

Except you can’t just blindly copy code snippets, you have to read the author’s explanation and perhaps adapt things to your own code sometimes, or reject their solution entirely. GPT-4 can’t do this because it doesn’t actually know what the hell it’s doing, it’s just putting stuff together in a form that is most probably correct based on what it has seen in training data for past examples.

I fear for the layman who sees a bunch of AI generated code and think it must be right. Who knows what bugs, security flaws, or performance issues they will run into, that they have no idea how to solve or even to begin asking a prompt for.

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#32

Post by ftufek »

alooPotato wrote: post #24
ftufek wrote: post #10

This is awesome, can't wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I'm thinking is, you can let the model decide the most relevant files. The initial prompt would be, "person wants to do x, here are the file list of this repo: ...., give me a list of files that you'd want to edit, create or delete" -> take the list, try to fit the con…

It's just so slow for the autocompletion use case to do it like that. Ideally, you're never chaining serial requests to the LLM. Even if you do stuff in all the data into a single prompt, the execution time seems to be superlinear with the number of tokens, again getting super slow.

Yeah I agree it's too slow for autocompletion at the moment, but this would be for full feature implementations, not just autocomplete. For example, if I have a repo I want to add a table and rest api implementation in, it can do this: https://imgur.com/a/mIJvaJr (ignore the formatting errors in the UI, somehow parts of it show up in as code and others not, but api wouldn't have this issue, especially since you can use the system message to enforce output format).

I'm happy to wait even 30-60 seconds for this which I can easily evaluate, criticize (and the model will correct it) and then proceed to just patch and move on. I think the results from this will be much better with the 32k model, but remains to be seen.

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#35

Post by WXLCKNO »

mpoon wrote: post #7
EGreg wrote: post #3

How much text can you feed GPT-4? Our codebase is 1 million lines of code. Can we feed the documentation to it? What are the limits? Is it possible to train it on our data without doing prompt engineering? How? Otherwise are we supposed to use embeddings? Can someone explain how these all work and the tradeoffs?

I'm waiting on my GPT-4 API access so I can use gpt-4-32k which maybe can soak up 10k LOC? Clearly this will break eventually, but I am playing around with some ideas to extend how much context I can give it. One is to do something like base64 encode file contents. I've seen some early success that GPT-4 knows how to decode it, so that'll allow me to stuff more characters into it. I'm also hoping that with the use of…

Have multiple instances of Gpt4 with different parts of the codebase interact with each other to write the whole thing.

Probably doesn't work this way lol

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#37

Post by layer8 »

xwdv wrote: post #13

Eventually you get to a point where the AI simply doesn’t know how to do something correctly. It can’t fix a certain bug, or implement a feature correctly. At this point you are left trying to do more and more prompt crafting… or you can just fix the problem yourself. If you can’t do it yourself, you’re screwed. I wonder if the future will just be software hobbled together with shitty AI code that no one understands,…

> software hobbled together with shitty AI code that no one understands

That’s my fear as well. Software may just get shittier overall (aka “good enough”), and in higher volumes, due to it taking less time to crank out using AI.

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#38

Post by wokwokwok »

ftufek wrote: post #10

This is awesome, can't wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I'm thinking is, you can let the model decide the most relevant files. The initial prompt would be, "person wants to do x, here are the file list of this repo: ...., give me a list of files that you'd want to edit, create or delete" -> take the list, try to fit the con…

Just remember the API charge is 6c per input token [1]. If you push 32k input tokens in, you're looking at $2000 per API call just as input.

You... might wanna consider a self hosted alternative for that use case, or at least do like, a `| wc` to get an idea of what you're potentially sending before calling the api.

[1] - https://help.openai.com/en/articles/7127956-how-much-does-gp...

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#39

Post by dangond »

wokwokwok wrote: post #38
ftufek wrote: post #10

This is awesome, can't wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I'm thinking is, you can let the model decide the most relevant files. The initial prompt would be, "person wants to do x, here are the file list of this repo: ...., give me a list of files that you'd want to edit, create or delete" -> take the list, try to fit the con…

Just remember the API charge is 6c per input token [1]. If you push 32k input tokens in, you're looking at $2000 per API call just as input. You... might wanna consider a self hosted alternative for that use case, or at least do like, a `| wc` to get an idea of what you're potentially sending before calling the api. [1] - https://help.openai.com/en/articles/7127956-how-much-does-gp...

6c per thousand tokens, so $2 per maxed out API call

Re: Show HN: GPT Repo Loader – load entire code repos into GPT prompts

#40

Post by [deleted] »

wokwokwok wrote: post #38
ftufek wrote: post #10

This is awesome, can't wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I'm thinking is, you can let the model decide the most relevant files. The initial prompt would be, "person wants to do x, here are the file list of this repo: ...., give me a list of files that you'd want to edit, create or delete" -> take the list, try to fit the con…

Just remember the API charge is 6c per input token [1]. If you push 32k input tokens in, you're looking at $2000 per API call just as input. You... might wanna consider a self hosted alternative for that use case, or at least do like, a `| wc` to get an idea of what you're potentially sending before calling the api. [1] - https://help.openai.com/en/articles/7127956-how-much-does-gp...

[deleted]
Post reply on HN