Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LearnDB: Learn how to build a database

learndb.net

Post reply on HN
Print view View original

11–20 of 56 posts

Re: LearnDB: Learn how to build a database

#11

Post by gtowey »

dymk wrote: post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

This was my thought as well. I'm curious to know if the author has previous experience building actual databases or if this is a case of follow-along-while-I-figure-this-out.

Re: LearnDB: Learn how to build a database

#12

Post by interesthrow2 »

johan_larson wrote: post #3
dominotw wrote: post #2

I realized how little i knew about how databases work until I watched this lecture series. https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA Does anyone have DB internal book recommendation that inst' boring as hell.

You might try this collection of papers about DB technology. http://www.redbook.io/ Fair warning: this is not for beginners. If you find the going too hard, start with a DB textbook. Also, that site doesn't seem to include the papers themselves. But most of those papers are very famous, so if you search for the titles, you should find copies. Worst case, you might need to visit a university library.

> Fair warning: this is not for beginners. If you find the going too hard, start with a DB textbook.

which one can teach how to build a DB system from scratch? I'm not talking about SQL theory or implementing a SQL parser but the actual persistence, indexing part.

Re: LearnDB: Learn how to build a database

#13

Post by emmanueloga_ »

rodw wrote: post #4

For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…

On the subject of readable database source code:

* LMDB is legendary for its performance, but it is source code is kind of hard to read. Found there's a pretty decent port to Java which looks a lot more readable [1].

* LMDB doesn't support SQL. But there's this SQL implementation: Apache Calcite [2]. Creating a DB using llmdbjava + Calcite sound like an interesting project.

1: https://github.com/lmdbjava/lmdbjava/tree/master/src/main/ja...

2: https://calcite.apache.org/

Re: LearnDB: Learn how to build a database

#14

Post by emmanueloga_ »

rodw wrote: post #4

For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…

Aside: found about your GraphViz cookbook on your site [1]. Looks super useful. Is it coming any time soon?

1: http://heyrod.com/projects/gv-cookbook.html

Re: LearnDB: Learn how to build a database

#15

Post by interesthrow2 »

dymk wrote: post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

> Why would a pedagogical project choose JavaScript of all languages?

I don't think the JS part is the issue. Node.js does I/O, files and co. The issue is that the article doesn't teach how to build a database at all.

It doesn't explain how to efficiently persist and fetch data from a file, indexing strategies with trees, concurrent file access,locking, basic transaction... that's what I expect from a tutorial about how to build a basic DB system.

Re: LearnDB: Learn how to build a database

#16

Post by codezero »

dymk wrote: post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

JavaScript is extremely accessible and the warts can easily be worked around.

I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t?

Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anything? I’m really curious, I’m not trying to set you up - I’m really not exeperienced enough to do so :)

Edit: to be clear because of general curiosity and because I hate bringing work home, most side projects I work on are in some other language.

Re: LearnDB: Learn how to build a database

#17

Post by dymk »

interesthrow2 wrote: post #15
dymk wrote: post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

> Why would a pedagogical project choose JavaScript of all languages? I don't think the JS part is the issue. Node.js does I/O, files and co. The issue is that the article doesn't teach how to build a database at all. It doesn't explain how to efficiently persist and fetch data from a file, indexing strategies with trees, concurrent file access,locking, basic transaction... that's what I expect from a tutorial about…

Plenty of languages do I/O, files, and co.

Why choose a language plagued with warts and a half-baked ecosystem for something pedagogical? A language with all sorts of peculiarities from the '95 browser era, and not at least leverage browser technology? A language which gives you very non-interesting coarse control over resource, and has no concept of parallelism?

Re: LearnDB: Learn how to build a database

#18

Post by thrower123 »

Mr_P wrote: post #5

How is this #2 on Hacker News? There's literally nothing here about how to actually build a database (yet?) Instead, there's just a key-value store implemented on top of a javascript hashmap and a filesystem.

I'd be very interested in a book or blog series similar to the scads of "how to build an os from scratch" or the raytracer projects, like Peter Shirley's little One Weekend series. Is there anything that scatches that itch?

Re: LearnDB: Learn how to build a database

#19

Post by dymk »

codezero wrote: post #16
dymk wrote: post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

JavaScript is extremely accessible and the warts can easily be worked around. I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t? Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anythin…

> and the warts can easily be worked around

Not really. The author doesn't leverage any kind of static typesystem, which would at least mitigate some of the warts. He's chosen Node.js instead of leveraging the browser, so no free visualization layer for doing anything interesting with, unfortunately.

> Have you worked with a JS language for any extended period of time?

Most of my day job is working in a really, really big Javascript codebase. I can say with confidence that the language is something that we're absolutely stuck with, and I still have no idea why somebody would implement a pedagogical database with it (well, then again, they haven't; they've implemented a key-value store, which is trivial).

To offer an alternative, they could have chosen something boring but everywhere like Java, which is just as accessible to those with less experience. Then they'd have the possibility of doing fine-grained parallelism, file access, designing abstractions that fit within a statically typed language.

Post reply on HN