Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AutoGPT engineers ditched vector databases

dariuszsemba.com

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Why AutoGPT engineers ditched vector databases

#51

Post by opdahl »

andre-z wrote: post #47
opdahl wrote: post #41

Earlier quoted context omitted.

I do about 100 million embeddings using around 50 GPU instances and feed them into Qdrant. Takes about 12 hours. Very happy with the result and performance as long as you have the option to have a very large memory instance running.

Hello from Qdrant. Would like to hear more about your use case. If not yet connected. https://www.linkedin.com/in/zayarni

Hey Andre. I've actually gotten a lot of great help talking to your co-founder Andrey on your discord channel and he helped me out a lot with making it work at our scale. Super happy with it and it's working in production with no hiccups.

Re: Why AutoGPT engineers ditched vector databases

#52

Post by penjelly »

amelius wrote: post #29
batmansmk wrote: post #14

« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this: Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas. In o…

Don't read the intermediate representation. The idea is that you use an LLM to summarize those comments into human readable text.

nobody else finds this ridiculous? That just means when i search for the actual source of truth (when llm eventually hallucinates) that it takes longer to find the answer im looking for. This fluff doesnt add any value for anyone, not even the LLM

Re: Why AutoGPT engineers ditched vector databases

#53

Post by penjelly »

benterix wrote: post #31

Has anyone ever managed to generate anything useful with AutoGPT? I had several attempts and apart from wasting some money for GPT-4 API calls, it's never produced anything usable. Whereas if I manually enter prompts in ChatGPT I can often produce simpler project from the beginning till the end, if I partition them into logically independent parts.

No.

Re: Why AutoGPT engineers ditched vector databases

#54

Post by dartos »

coffeebeqn wrote: post #43
datadrivenangel wrote: post #5

Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option. But that never got anyone promoted.

So it seems like they still use vectors - they just replaced the search (however that works) with a dot product operation? I mean from a vector point of view that makes total sense

Searching in LLM land is finding the cosine similarity of two high dimensional vectors. Vector databases try optimizing that operation.

Maybe they found search by plain old dot product faster

Re: Why AutoGPT engineers ditched vector databases

#55

Post by dartos »

benterix wrote: post #31

Has anyone ever managed to generate anything useful with AutoGPT? I had several attempts and apart from wasting some money for GPT-4 API calls, it's never produced anything usable. Whereas if I manually enter prompts in ChatGPT I can often produce simpler project from the beginning till the end, if I partition them into logically independent parts.

Me neither.

Re: Why AutoGPT engineers ditched vector databases

#56

Post by tudorw »

andai wrote: post #48
tudorw wrote: post #25

Earlier quoted context omitted.

It can do stuff, sort of, like helping me to create ; https://github.com/tudorw/Ai_MegaList/blob/main/AI_Sector_Br... After a lot of trial and error, I managed to keep it somewhat on track by using a CSV file, something like; "An expert manipulate .csv files, read the first URL from the first line, 2nd column of 'raw.csv', pass the URL to browse_website the questions 'summarize the activities, highlight any investmen…

Are you referring to Waggle Dance or AutoGPT?

AutoGPT

Re: Why AutoGPT engineers ditched vector databases

#57

Post by akomtu »

dartos wrote: post #54
coffeebeqn wrote: post #43

Earlier quoted context omitted.

So it seems like they still use vectors - they just replaced the search (however that works) with a dot product operation? I mean from a vector point of view that makes total sense

Searching in LLM land is finding the cosine similarity of two high dimensional vectors. Vector databases try optimizing that operation. Maybe they found search by plain old dot product faster

Cosine what? Isn't it just dot product?

Re: Why AutoGPT engineers ditched vector databases

#58

Post by uoaei »

batmansmk wrote: post #14

« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this: Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas. In o…

Ironic that boilerplate code is removed but boilerplate copy is introduced to replace it.

Re: Why AutoGPT engineers ditched vector databases

#59

Post by dartos »

akomtu wrote: post #57
dartos wrote: post #54

Earlier quoted context omitted.

Searching in LLM land is finding the cosine similarity of two high dimensional vectors. Vector databases try optimizing that operation. Maybe they found search by plain old dot product faster

Cosine what? Isn't it just dot product?

There are a few “distance” metrics that are used.

AFAIK cosine similarity or cosine distance is a common one bc it’s faster than a dot product.

Post reply on HN