This is great; we will try it out. I will get on the Slack in the coming days to have a chat!
That's amazing!, Please try out the Jupyter demos :)
Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
21–30 of 36 posts
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#22So I have 1.6M rows of title, body and author. Can I use SDDB to embed all rows with a local model and then do semantic search as a query? And how can this scale past a single machine?
Thanks for commenting :)
Absolutely you can query the embeddings of 1.6M rows, you can try with lance or in memory vector search type. The scalablity will depends on embedding size, machine configurations, etc. Thanks
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#23I think this is very smart. Instead of a big ML ops stack, do your model stuff where your data is. For us, the fact it supports Postgres and SQLite out of the box is awesome. It means we can run a model on the server to generate embeddings, sync these to a local SQLite and then run local inference on the same data structure.
Thanks for commenting :),
Sure you can have a mesh of database, Have your input data in one data technology, run models on a distributed compute, save models on different database tech and manage the jobs on other!
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#24This looks like a decent project, but I really didn't appreciate waking up to messages from mods on my community notifying me that you joined lastnight, ignored my welcome message, request of giving us an intro to yourself and what you're working on, didn't bother reading the community rules and then just blatantly started spamming my users to upvote you on ProductHunt.
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#25Congrats on the launch, it looks like you worked very hard on it.
But I’m an engineer, I read the README and the website, and I still don’t know what Super-duper is.
Is it just a python library? Does it have its own persistence (it must)? It doesn’t appear to be a set of plugins for various DB’s but I could be wrong.
As such I don’t know how I’d use it. It might be helpful to describe the product in more concrete terms.
One of the phrases used in YC is: ACME makes soup taste better. We do it with a seasoning that chefs add to their broth.
Maybe that’s helpful. Explaining a product can be hard!
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#26Congrats on the launch, it looks like you worked very hard on it. But I’m an engineer, I read the README and the website, and I still don’t know what Super-duper is . Is it just a python library? Does it have its own persistence (it must)? It doesn’t appear to be a set of plugins for various DB’s but I could be wrong. As such I don’t know how I’d use it. It might be helpful to describe the product in more concrete te…
By connecting AI models with the data's source (the database) we make it very easy to bring AI to your end-user-facing applications.
SuperDupeDB is really an end-to-end AI development and deployment framework wrapping and integrating your existing data infrastructure. It replaces MLOps entirely as it covers inference and model training.
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#27Congrats on the launch, it looks like you worked very hard on it. But I’m an engineer, I read the README and the website, and I still don’t know what Super-duper is . Is it just a python library? Does it have its own persistence (it must)? It doesn’t appear to be a set of plugins for various DB’s but I could be wrong. As such I don’t know how I’d use it. It might be helpful to describe the product in more concrete te…
All things aside, it's a framework for building data workflows in Python.
Like taking the data from that source (e.g., SQL), processing them (e.g., pytorch or openai), and storing the results somewhere (e.g., data on Mongo metadata on SQL).
It actually consists of the following: 1. nifty abstractions for Data (e.g., sources, encoders, listeners), Metadata (e.g., vector indexes), Compute (e.g., sync, async, parallel). 2. gluing engine that transparently handles the interaction between components 3. out-of-the-box integrations with established tools (databases, AI models and APIs, compute engines)
This way, you can build customized data layers that sit on top of your database and save you from moving the data to dedicated systems (e.g., vector databases or MLops tools)
For further discussion, feel free to join our slack https://join.slack.com/t/superduperdb/shared_invite/zt-1zuoj...
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#28This project just sent me unsolicited DM spam on a Slack for a tangentially related project. I strongly encourage avoiding this project as they are using spam and harassment to promote their project. From the sounds of another comment on this thread, the community this project harassed me in is not the only community they have done this in this morning.
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#29This project just sent me unsolicited DM spam on a Slack for a tangentially related project. I strongly encourage avoiding this project as they are using spam and harassment to promote their project. From the sounds of another comment on this thread, the community this project harassed me in is not the only community they have done this in this morning.
[deleted]
Re: Show HN: SuperDuperDB – Open-source framework for integrating AI with databases
#30Congrats on the launch, it looks like you worked very hard on it. But I’m an engineer, I read the README and the website, and I still don’t know what Super-duper is . Is it just a python library? Does it have its own persistence (it must)? It doesn’t appear to be a set of plugins for various DB’s but I could be wrong. As such I don’t know how I’d use it. It might be helpful to describe the product in more concrete te…
All things aside, it's a framework for building data workflows in Python. Like taking the data from that source (e.g., SQL), processing them (e.g., pytorch or openai), and storing the results somewhere (e.g., data on Mongo metadata on SQL). It actually consists of the following: 1. nifty abstractions for Data (e.g., sources, encoders, listeners), Metadata (e.g., vector indexes), Compute (e.g., sync, async, parallel).…
Ah, it’s a langchain competitor, possibly with better DB support.
One of the nice things about langchain is the code examples, making it easy to get simple services up and running. And because it’s a toolkit I can take what I need and leave the rest.
However, the ecosystem around langchain is really exploding, is there some way you can retool what you have to extend langchain with better DB support, rather than build your own thing?