Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Greptile (YC W24) - RAG on codebases that actually works

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 173 posts

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#81

Post by dvt »

Ran it on a "real" OSS project of mine (https://github.com/dvx/lofi/), and it was stuck at 99% loading for about 30 minutes. Then, when it finally parsed the codebase, when asked anything it always returns "Error: Internal error while locating sources." Specifically, I wanted to see if it can context switch between TypeScript (used for the front-end), ObjectiveC (used for a few Mac features), C++ (used for Windows volume features), and GLSL (used for visualizations). But alas.

At one point, this random prompt popped up: https://imgur.com/a/mYeluaU —what's "Onboard?" Is this some kind of weird LLM leakage/hallucination?

With all respect, this is like a pre-MVP quality product. The codebase isn't even particularly large and the experience is extremely sub-par. Charging for something like this is honestly highway robbery.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#82

Post by yunwal »

jaffee wrote: post #78
simonw wrote: post #76

"We don't store any code on our servers after initial processing" Are you storing the embedding vectors you've calculated from the code? If so, those are likely quite easily reversible - so I would still consider that source code stored on your servers from the point of view of a security audit. As a result, I might actually prefer to have copies of my code stored on your servers if it resulted in faster performance.

> embedding vectors you've calculated from the code? If so, those are likely quite easily reversible I don't think embeddings are generally reversible... you're usually projecting onto a lower dimensional space, and therefore losing information.

"Quite easily" isn't true in most cases, but embeddings are sometimes reversible. We know this because programs like Stable Diffusion sometimes output near-perfect copies of training data when given the correct prompt, and generation of that image is based on word and image embeddings alone.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#83

Post by dartos »

jaffee wrote: post #78
simonw wrote: post #76

"We don't store any code on our servers after initial processing" Are you storing the embedding vectors you've calculated from the code? If so, those are likely quite easily reversible - so I would still consider that source code stored on your servers from the point of view of a security audit. As a result, I might actually prefer to have copies of my code stored on your servers if it resulted in faster performance.

> embedding vectors you've calculated from the code? If so, those are likely quite easily reversible I don't think embeddings are generally reversible... you're usually projecting onto a lower dimensional space, and therefore losing information.

I’ve never heard of reversible embeddings in practice.

In theory if you know the model being used you could reverse them.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#84

Post by soohoonchoi »

simonw wrote: post #76

"We don't store any code on our servers after initial processing" Are you storing the embedding vectors you've calculated from the code? If so, those are likely quite easily reversible - so I would still consider that source code stored on your servers from the point of view of a security audit. As a result, I might actually prefer to have copies of my code stored on your servers if it resulted in faster performance.

For now we are storing the embeddings of the generated docstrings, the intuition behind this is kinda like how HyDE works. I don't think the actual code itself is reversible from the embeddings, but yes we do want to store the actual code at some point and we will need the proper security measures for that. We are also actively developing an self-hostable version for enterprise.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#85

Post by msikora »

koeng wrote: post #38

I'd love to try it, but pretty much all my repos are >10mb. It's not because there is that much code, but because I am doing bioinformatics and the test files (for the unit tests) inflate the repo size. It would be great if there was a way to test it on just 1 large repo for perhaps a week or something, because I balk at the idea of spending $20 a month on something that I don't even know works well. This is importan…

Maybe we need a `.greptileignore` file...

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#86

Post by jncraton »

jaffee wrote: post #78
simonw wrote: post #76

"We don't store any code on our servers after initial processing" Are you storing the embedding vectors you've calculated from the code? If so, those are likely quite easily reversible - so I would still consider that source code stored on your servers from the point of view of a security audit. As a result, I might actually prefer to have copies of my code stored on your servers if it resulted in faster performance.

> embedding vectors you've calculated from the code? If so, those are likely quite easily reversible I don't think embeddings are generally reversible... you're usually projecting onto a lower dimensional space, and therefore losing information.

You might be interested in "Text Embeddings Reveal (Almost) As Much As Text":

> We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.

https://arxiv.org/pdf/2310.06816.pdf

There's certainly information loss, but there is also a lot of information still present.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#87

Post by simonw »

jncraton wrote: post #86
jaffee wrote: post #78

Earlier quoted context omitted.

> embedding vectors you've calculated from the code? If so, those are likely quite easily reversible I don't think embeddings are generally reversible... you're usually projecting onto a lower dimensional space, and therefore losing information.

You might be interested in "Text Embeddings Reveal (Almost) As Much As Text": > We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. https://arxiv.org/pdf/2310.06816.pdf There's certainly information loss, but there is also a lot of information still present.

Yeah, that paper is what I was thinking about. https://simonwillison.net/2024/Jan/8/text-embeddings-reveal-...

“a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly”.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#88

Post by Axsuul »

drcongo wrote: post #27

I've been looking for something like this, but local-only. Any plans to let people self-host and point at local repositories?

Also looking for something local but I feel like Apple will probably eventually release local LLMs for your entire filesystem.

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#89

Post by soohoonchoi »

dvt wrote: post #81

Ran it on a "real" OSS project of mine ( https://github.com/dvx/lofi/ ), and it was stuck at 99% loading for about 30 minutes. Then, when it finally parsed the codebase, when asked anything it always returns "Error: Internal error while locating sources." Specifically, I wanted to see if it can context switch between TypeScript (used for the front-end), ObjectiveC (used for a few Mac features), C++ (used for Windows…

Hey sorry to hear that, a couple of things:

- processing is usually stuck at 99% because when we order the components of the repository by file-directory dependency and ast dependency, there are a lot, LOT more leaves than the internal nodes + the root. Since we have to have the results of the dependency before we move on, moving up the dependency chain with llm calls take a while. This is even more pronounced when nearing the root for completion. We are working on optimizing this flow as it is very annoying for us as well.

- "Error: Internal error while locating sources": this is embarrassing but we did experience a database outage today (I wonder why) some repositories have a faulty status. It should be back up now and we are working to recover/reprocess the repos that have failed during the outage (including dvx/lofi)

- "Onboard" that was our previous name, this has slipped through the cracks, thanks for pointing it out!

We are trying to parse most of the popular open source repos (we have processed repos like python, vscode, etc). We are hoping to fully process the linux kernel soon as well (a personal benchmark of mine).

Re: Launch HN: Greptile (YC W24) - RAG on codebases that actually works

#90

Post by dakshgupta »

msikora wrote: post #85
koeng wrote: post #38

I'd love to try it, but pretty much all my repos are >10mb. It's not because there is that much code, but because I am doing bioinformatics and the test files (for the unit tests) inflate the repo size. It would be great if there was a way to test it on just 1 large repo for perhaps a week or something, because I balk at the idea of spending $20 a month on something that I don't even know works well. This is importan…

Maybe we need a `.greptileignore` file...

That's funny, we actually do ignore those files but realizing now we don't account for that in our calculation of codebase size since we just get that via the GH API.
Post reply on HN