There's already an (albeit deprecated) debugger package called pydb (http://bashdb.sourceforge.net/pydb/). It would be good to choose a different name for this, most importantly because `pip install pydb` is already taken
Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
11–20 of 34 posts
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#12I wrote a similar library -BlitzDB- a while ago:
https://github.com/adewes/blitzdb
It's a pure Python database engine with a MongoDB-like query engine and support for three different backends: File (native), SQL (via SQLAlchemy) and MongoDB.
The library transparently translates a large number of MongoDB queries into SQL or its own native storage backend, and when using the SQL backend it can do things that MongoDB can't, like queries spanning multiple relationships.
The latest version is not fully documented yet but I'm using it on several production projects myself. I'm looking for a maintainer and contributors btw, so if you're interested feel free to get in touch with me!
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#13[deleted]
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#14There's already an (albeit deprecated) debugger package called pydb ( http://bashdb.sourceforge.net/pydb/ ). It would be good to choose a different name for this, most importantly because `pip install pydb` is already taken
Thank you! I looked up databases to see if there were obvious name clashes but never thought of debuggers despite using `pdb` a lot.
I'm open to suggestions for names if anyone has one. (Although I guess it'd be nice if the top discussion thread wasn't about naming.)
EDIT: I just saw you opened an issue on github so I'm happy to take name suggestions there.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#15What's the purpose/gain of layering ZMQ into this? I read the architecture bit but I'm still unclear as to what benefit this brings. I guess it allows for multiple clients to use the database at the same time? I can see how the queueing thing is useful for writes if you don't want to have to handle more than one at the same time for the sake of complexity, but wouldn't doing this for reading slow things down unnecess…
As you correctly observed, I mainly used ZeroMQ for the fan-in, so I need only consider one request at a time without worrying about chunking, disconnection or other lower (socket) level issues.
For speed, the idea is that you could potentially have multiple read-only servers answering queries simultaneous (all taking from the dealer). This isn't fleshed out yet. It possibly involves splitting requests into two queues for read and write requests (instead of only "run").
I'd be interested in hearing any info about potential slowdowns if you have them.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#16It might not hold up to heavy use cases, but I can already imagine a bunch of ways this would make my life easier (I use Python scripts to handle a bunch of social media stuff and basic analytics, for instance, where I use either text files or some other proxy to handle state.) The only request I'd have is a syntactically saccharine way of spinning up the server within the client itself, which is in general an awful…
Thanks! I'm glad to see this looks potentially useful to someone other than myself.
If the client could start servers (other than by importing from server.py), what would be the intended outcome if two clients try to spin up servers at the same time? Otherwise would a file lock do? In that case, just file locking combined with pickle could possibly be enough for your needs.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#17You could use `-e git://github.com/asrp/undoable` in requirements.txt to save some steps.
Oh, thanks, I wasn't aware of that flag. Though for the moment
pip install -e git://github.com/asrp/undoable#egg=undoable
I'll probably look into that soon.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#18It might not hold up to heavy use cases, but I can already imagine a bunch of ways this would make my life easier (I use Python scripts to handle a bunch of social media stuff and basic analytics, for instance, where I use either text files or some other proxy to handle state.) The only request I'd have is a syntactically saccharine way of spinning up the server within the client itself, which is in general an awful…
Thanks! I'm glad to see this looks potentially useful to someone other than myself. If the client could start servers (other than by importing from server.py), what would be the intended outcome if two clients try to spin up servers at the same time? Otherwise would a file lock do? In that case, just file locking combined with pickle could possibly be enough for your needs.
+1 (I find it useful too). I actually started writing my own implementation of something similar (named "PersistendDict" - feel free to steal the name ;) ) but never finished it so I will definitely check out your library. Thanks!
On account of starting / stopping the server... I personally don't care if functionality is there or not as I can always start server manually on dev machine, and it should be started differently on production machines anyway. But that's just my opinion.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#19There's already an (albeit deprecated) debugger package called pydb ( http://bashdb.sourceforge.net/pydb/ ). It would be good to choose a different name for this, most importantly because `pip install pydb` is already taken
Reminds me of the time I lost a solid hour because the pip repository name was different than the module name.
Re: Show HN: Pydb – a lightweight database with Python syntax queries, using ZeroMQ
#20You could use `-e git://github.com/asrp/undoable` in requirements.txt to save some steps.
Oh, thanks, I wasn't aware of that flag. Though for the moment pip install -e git://github.com/asrp/undoable#egg=undoable tells me `setup.py` doesn't exist (because it doesn't). I haven't gotten around to packaging yet not knowing (before today) if anyone's interested. I'll probably look into that soon.
I decided to push a quick patch to github so installation with just `pip install -r requirements.txt` is possible now. I'll look into proper packaging later on. Thanks again!