Show HN: Piazza, syncing the web in a vectorized DB

#1

Post by lcombaldieu » Thu, Dec 05, 2024, 2:41 PM UTC

We recently released an open source project that syncs wikipedia with a vector database : https://github.com/Piazza-tech/Piazza-Updater

We used Verba, Weaviate and Docker for deployment

We'd like to have some feedback on how to continue the project, which data sources would be interesting to vectorize. You can give feedback on our landing page http://piazza.tech Please leave a star !

Show HN: Piazza, syncing the web in a vectorized DB
piazza.tech

Re: Show HN: Piazza, syncing the web in a vectorized DB

#2

Post by CryptoBanker » Fri, Dec 06, 2024, 3:21 PM UTC

Demo doesn’t appear to be working

Re: Show HN: Piazza, syncing the web in a vectorized DB

#3

Post by Cheer2171 » Fri, Dec 06, 2024, 3:21 PM UTC

Might want to find a new name. Piazza (https://piazza.com) is a very popular classroom Q&A and wiki platform.

Re: Show HN: Piazza, syncing the web in a vectorized DB

#4

Post by igorguerrero » Fri, Dec 06, 2024, 3:24 PM UTC

I'm not sure if I'd call RAG an "advanced technique"...

Re: Show HN: Piazza, syncing the web in a vectorized DB

#5

Post by costco » Fri, Dec 06, 2024, 5:21 PM UTC

> You've requested a page on a website (chat.piazza.tech) that is on the Cloudflare network. The host (chat.piazza.tech) is configured as an Argo Tunnel, and Cloudflare is currently unable to resolve it.

Re: Show HN: Piazza, syncing the web in a vectorized DB

#6

Post by tejaskumar_ » Sat, Dec 07, 2024, 7:34 AM UTC

Interesting project! Reminds me of Astra DB who just worked with Wikimedia to vectorize Wikipedia: https://blog.wikimedia.de/2024/09/17/wikidata-and-artificial...

Nice work on the open source approach! Looking forward to seeing how this evolves.

Re: Show HN: Piazza, syncing the web in a vectorized DB

#7

Post by marco_z » Sun, Dec 08, 2024, 5:23 AM UTC

costco wrote: post #5

> You've requested a page on a website (chat.piazza.tech) that is on the Cloudflare network. The host (chat.piazza.tech) is configured as an Argo Tunnel, and Cloudflare is currently unable to resolve it.

same for me now.

Re: Show HN: Piazza, syncing the web in a vectorized DB

#8

Post by bosky101 » Mon, Dec 09, 2024, 7:47 AM UTC

I see many 3rd party deps, integrations for verba(first stumbling on this rag chat+frontend)

Can you give the stack that is suited well for completely self hosting piazza? Will search for those env and docs accordingly

Re: Show HN: Piazza, syncing the web in a vectorized DB

#9

Post by lcombaldieu » Mon, Dec 09, 2024, 12:00 PM UTC

bosky101 wrote: post #8

I see many 3rd party deps, integrations for verba(first stumbling on this rag chat+frontend) Can you give the stack that is suited well for completely self hosting piazza? Will search for those env and docs accordingly

OP here, are you asking about hardware or cloud instance type ? I would recommend using GPUs at least to initialise the DB. You need a lot of compute once at the start to vectorize all the data, and then regularly (but less intensely) to sync the updated changes. You can check out the Weaviate docs here if you want more info : https://weaviate.io/developers/weaviate