Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

github.com

Post reply on HN
Print view View original

31–40 of 82 posts

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#31

Post by memefrog »

kindawinda wrote: post #19
kindawinda wrote: post #2

Okay this seems like the perfect project to test out the various AI repo update solutions. Anyone want to do this for fun, you can have your own agenda, I'm just bored! https://github.com/paul-gauthier/aider https://www.mentat.codes/ https://www.gitwit.dev/ https://www.second.dev/

Why would you downvote this? Ive already turned it into a viable sqlite competitor. Shame on you.

Because you people that bring up the latest overhyped and useless buzzword nonsense on every single post are super tiring.

We do not need a comment about "someone should apply AI/ML to this" on every post any more than we need a "this but NFT" or "this but blockchain" or "this but React" or "this but OOP" comment. Your comment is just noise.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#32

Post by semiquaver »

OJFord wrote: post #20
simonw wrote: post #13

Thanks to this post I learned about Lark, which looks like a really nice parser library for Python. The JSON tutorial on their site is excellent - shows how to build a basic parser for JSON, then goes into some great detail about how to improve its performance: https://lark-parser.readthedocs.io/en/latest/json_tutorial.h... Here's the grammar used for the RDBMS project: https://github.com/spandanb/learndb-py/blob/mas…

DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that? Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow. This is just first thoughts at a glance, maybe I'm missing someth…

It is https://en.m.wikipedia.org/wiki/Backus–Naur_form or something similar. A python DSL would certainly be less expressive and useful.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#33

Post by spanspan »

OJFord wrote: post #20
simonw wrote: post #13

Thanks to this post I learned about Lark, which looks like a really nice parser library for Python. The JSON tutorial on their site is excellent - shows how to build a basic parser for JSON, then goes into some great detail about how to improve its performance: https://lark-parser.readthedocs.io/en/latest/json_tutorial.h... Here's the grammar used for the RDBMS project: https://github.com/spandanb/learndb-py/blob/mas…

DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that? Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow. This is just first thoughts at a glance, maybe I'm missing someth…

̶T̶h̶i̶s̶ ̶s̶t̶r̶i̶n̶g̶ ̶i̶s̶ ̶t̶h̶e̶ ̶g̶r̶a̶m̶m̶a̶r̶ ̶t̶h̶a̶t̶ ̶l̶a̶r̶k̶ ̶u̶s̶e̶s̶ ̶t̶o̶ ̶p̶a̶r̶s̶e̶ ̶t̶h̶e̶ ̶u̶s̶e̶r̶ ̶s̶u̶b̶m̶i̶t̶t̶e̶d̶ ̶s̶q̶l̶ ̶i̶n̶t̶o̶ ̶a̶n̶ ̶A̶S̶T̶.̶ ̶T̶h̶i̶s̶ ̶p̶a̶r̶s̶i̶n̶g̶ ̶i̶s̶ ̶f̶a̶r̶ ̶m̶o̶r̶e̶ ̶c̶o̶m̶p̶l̶e̶t̶e̶ ̶a̶n̶d̶ ̶r̶o̶b̶u̶s̶t̶ ̶t̶h̶a̶n̶ ̶w̶h̶a̶t̶ ̶d̶o̶i̶n̶g̶ ̶t̶h̶i̶s̶ ̶i̶n̶ ̶p̶u̶r̶e̶ ̶p̶y̶t̶h̶o̶n̶ ̶w̶o̶u̶l̶d̶ ̶a̶l̶l̶o̶w̶ ̶(̶w̶i̶t̶h̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶u̶r̶s̶e̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶i̶n̶g̶ ̶t̶h̶e̶ ̶e̶n̶t̶i̶r̶e̶ ̶l̶e̶x̶e̶r̶ ̶a̶n̶d̶ ̶p̶a̶r̶s̶e̶r̶ ̶t̶h̶a̶t̶ ̶l̶a̶r̶k̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶s̶)̶.̶

T̶h̶e̶r̶e̶ ̶m̶a̶y̶ ̶b̶e̶ ̶s̶o̶m̶e̶ ̶p̶o̶s̶t̶ ̶p̶a̶r̶s̶i̶n̶g̶ ̶v̶a̶l̶i̶d̶a̶t̶i̶o̶n̶ ̶t̶h̶a̶t̶ ̶c̶a̶n̶ ̶b̶e̶ ̶d̶o̶n̶e̶ ̶h̶e̶r̶e̶-̶ ̶b̶u̶t̶ ̶t̶h̶a̶t̶ ̶w̶o̶u̶l̶d̶ ̶b̶e̶ ̶s̶o̶m̶e̶t̶h̶i̶n̶g̶ ̶t̶h̶a̶t̶'̶s̶ ̶b̶e̶y̶o̶n̶d̶ ̶t̶h̶e̶ ̶d̶o̶m̶a̶i̶n̶ ̶o̶f̶ ̶t̶h̶e̶ ̶p̶a̶r̶s̶e̶r̶.̶

Edit: I see what you mean. I surveyed a bunch a parser generator libraries, and they also seemed to use a text based DSL- rather than DSL based on python structures. What you're describing would have made the grammar development more ergonomic and simple.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#34

Post by spanspan »

schemescape wrote: post #25

Does it support any ACID guarantees? Query planning/optimization? I’m not asking this to imply that it can/should. I just want to know what all was attempted besides b-trees and SQL. I’d like to do something like this someday. Nice work!

Re: ACID guarantees

It doesn't have a notion of atomically batching multiple statements, i.e. transaction. But beyond that, it's a single file database, which can only have a single process (learndb instance) that is operating on the database (file). So you get consistency and isolation via being a single connection database. Durability, you get to the extent that the file system is durable. So it's somewhere on the ACIDity spectrum.

Re: Query planning/optimization

I haven't implemented this; but I've considered where the optimization could module sit: The parser spits out an AST. This or a derived intermediate representation could be optimized,i.e. the AST could be rewritten or nodes deleted, before the VM executes the AST.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#35

Post by OJFord »

semiquaver wrote: post #32
OJFord wrote: post #20

Earlier quoted context omitted.

DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that? Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow. This is just first thoughts at a glance, maybe I'm missing someth…

It is https://en.m.wikipedia.org/wiki/Backus–Naur_form or something similar. A python DSL would certainly be less expressive and useful.

I realise that (with some additions, certainly - it has imports) I just don't think embedding it in a string is 'really nice', personally. The 'developer experience' will be crap or reliant on hyper-specific tooling; so likely crap. (The RHS of that 'or' will be too niche to be good and well-maintained, probably.)

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#36

Post by OJFord »

tmpX7dMeXU wrote: post #26
OJFord wrote: post #20

Earlier quoted context omitted.

DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that? Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow. This is just first thoughts at a glance, maybe I'm missing someth…

It’s an existing, standard, language for describing grammars. Quite clearly the trade-off being made here. Seemingly not the one you would’ve made. Don’t act like it’s objectively bad.

in a string. I didn't say the DSL was novel. I learnt BNF too. (It does have extras that are either unique to it or standard beyond my familiarity though.)

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#38

Post by malkia »

Long long time ago someone rewrote/ported SQLite from C to C# -> https://code.google.com/archive/p/csharp-sqlite/wikis/Letter... - note how welcome Dr. Richard Hipp was on the effort!

Probably here on github -> https://github.com/CsharpDatabase/CsharpSQLite - and possibly some more clones after that.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#40

Post by hluska »

keithalewis wrote: post #29
spanspan wrote: post #18

Earlier quoted context omitted.

Single file, embedded database with similar logical organization

Perhaps a more accurate claim would be "SQLite inspired". Calling it a clone is misleading. Mad props to the author. Many Python programmers never had proper training in computer science, so it is encouraging to see people filling in the gaps of their knowledge.

This is a very early release, whereas SQLite has 22 years of releases. In that light, this is about the least charitable take on this.

Someone in our community built something and had the courage to release it. Your criticism is unfair.

Post reply on HN