Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

github.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#41

Post by jarulraj »

simonw wrote: post #13

Thanks to this post I learned about Lark, which looks like a really nice parser library for Python. The JSON tutorial on their site is excellent - shows how to build a basic parser for JSON, then goes into some great detail about how to improve its performance: https://lark-parser.readthedocs.io/en/latest/json_tutorial.h... Here's the grammar used for the RDBMS project: https://github.com/spandanb/learndb-py/blob/mas…

Highly recommend Lark for Python projects -- it's easy to use :)

Their IDE was super useful for debugging the grammar: https://www.lark-parser.org/ide/

We use Lark for a SQL-like language tailored for using AI models in EvaDB: https://github.com/georgia-tech-db/evadb/blob/master/evadb/p... https://github.com/georgia-tech-db/evadb/

If you like Lark, please consider sponsoring them: https://github.com/sponsors/lark-parser

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#43

Post by gabereiser »

fuzztester wrote: post #12
devman0 wrote: post #8

Earlier quoted context omitted.

This is why you see a community of pure Java RDBMSs (Hypersonic, H2, Derby, etc), if you don't need the big iron scale it's easier to ship/use the database or even embed it in memory if needed.

Yes. It's an interesting area, irrespective of implementation language. I had tried out one or two Java-based RDBMSes back in the day, via programs written in Java, for fun. I think one was HSQLDB. https://en.m.wikipedia.org/wiki/HSQLDB There was also another interesting one called PointBase, which was developed by Bruce Scott, an Oracle founder, and others. https://en.m.wikipedia.org/wiki/PointBase

H2 is pretty great and fully supports JSONB and the like [1]. Full java-only RDBMS.

[1]: https://www.h2database.com/html/main.html

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#44

Post by esjeon »

hluska wrote: post #40
keithalewis wrote: post #29

Earlier quoted context omitted.

Perhaps a more accurate claim would be "SQLite inspired". Calling it a clone is misleading. Mad props to the author. Many Python programmers never had proper training in computer science, so it is encouraging to see people filling in the gaps of their knowledge.

This is a very early release, whereas SQLite has 22 years of releases. In that light, this is about the least charitable take on this. Someone in our community built something and had the courage to release it. Your criticism is unfair.

> Your criticism is unfair.

I think it doesn't even get close to being a criticism, and it's certainly unclear if the goal is to literally clone SQLite or to implement SQLite-ish. This is a fair question.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#45

Post by spanspan »

jarulraj wrote: post #42

Really cool project! I am curious about the benefits and limitations of using Python in this project as opposed to C++. How well is LearnDB able to support low-level concurrency control and storage management?

I began using python as a way to "mock" out the overall design; intending to re-implement it in rust. The main reasoning for using python: was the ability to focus on "high-level" concepts and speed of tinkering. This implements a single process, single thread, single connection database- so performance and low-level concurrency control were not explicit goals or really optimized for. For those (real-live concerns) rust or C++ are much better; but also come with their set of complexities.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#46

Post by keithalewis »

hluska wrote: post #40
keithalewis wrote: post #29

Earlier quoted context omitted.

Perhaps a more accurate claim would be "SQLite inspired". Calling it a clone is misleading. Mad props to the author. Many Python programmers never had proper training in computer science, so it is encouraging to see people filling in the gaps of their knowledge.

This is a very early release, whereas SQLite has 22 years of releases. In that light, this is about the least charitable take on this. Someone in our community built something and had the courage to release it. Your criticism is unfair.

Just trying to encourage clear and accurate communication. I agree with your first sentence. The only thing unfair is the author claiming it is a SQLite clone. It isn't, as we both seem to agree. It is a form of cheating.

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#47

Post by [deleted] »

OJFord wrote: post #36
tmpX7dMeXU wrote: post #26

Earlier quoted context omitted.

It’s an existing, standard, language for describing grammars. Quite clearly the trade-off being made here. Seemingly not the one you would’ve made. Don’t act like it’s objectively bad.

in a string . I didn't say the DSL was novel. I learnt BNF too. (It does have extras that are either unique to it or standard beyond my familiarity though.)

[deleted]

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#48

Post by [deleted] »

simonw wrote: post #13

Thanks to this post I learned about Lark, which looks like a really nice parser library for Python. The JSON tutorial on their site is excellent - shows how to build a basic parser for JSON, then goes into some great detail about how to improve its performance: https://lark-parser.readthedocs.io/en/latest/json_tutorial.h... Here's the grammar used for the RDBMS project: https://github.com/spandanb/learndb-py/blob/mas…

[deleted]

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#49

Post by spanspan »

keithalewis wrote: post #46
hluska wrote: post #40

Earlier quoted context omitted.

This is a very early release, whereas SQLite has 22 years of releases. In that light, this is about the least charitable take on this. Someone in our community built something and had the courage to release it. Your criticism is unfair.

Just trying to encourage clear and accurate communication. I agree with your first sentence. The only thing unfair is the author claiming it is a SQLite clone. It isn't, as we both seem to agree. It is a form of cheating.

Fair. It’s “inspired by”, not a “clone”. Frankly, I don’t think these terms are that specific, that one couldn’t level the same point against “inspired by”.. in what sense is it inspired?

Re: Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python

#50

Post by dleslie »

simonw wrote: post #30
OJFord wrote: post #20

Earlier quoted context omitted.

DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that? Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow. This is just first thoughts at a glance, maybe I'm missing someth…

That's pretty standard for parsing libraries - have you seen any good ones (for Python or other languages) that don't use a DSL like this? The only one I've seen is this one: https://parsy.readthedocs.io/en/latest/tutorial.html

Parse, comparse, parsec and PEGs come to mind.

1: https://github.com/massung/parse

2: https://wiki.call-cc.org/eggref/5/comparse

3: https://hackage.haskell.org/package/parsec

4: https://janet-lang.org/docs/peg.html

Post reply on HN