Live data from Hacker News

LearnDB: Learn how to build a database

learndb.net

21–30 of 56 posts

Re: LearnDB: Learn how to build a database

#21
post #6
post #5

How is this #2 on Hacker News? There's literally nothing here about how to actually build a database (yet?) Instead, there's just a key-value store implemented on top of a javascript hashmap and a filesystem.

This has to be a joke? This looks more like 'how to use a k-v store 101 for non programmers' I don't want to put more content under a terrible post, but the best resource for this material Jennifer Widom's MOOC. https://cs.stanford.edu/people/widom/DB-mooc.html

I took this back in 2011 and don’t remember there being much on “building a database”. Is there more advanced content now?

Re: LearnDB: Learn how to build a database

#22
post #4

For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…

Yup, a good way to understand why RDBMS are they way they are is to try to make your own in whatever language you choose. I've done this on a very elementary scale making a built in DB for a past project, at first it was single threaded, but when you start considering indexing and the datas structures necessary, it makes you understand and appreciate the man hours that have gone into projects like SQLite, PostgreSQL etc. At first you think, oh I can just write my own database, but when you start getting into areas and the CAP theorem starts to create problems, you really understand why anything past a local machine only ephemeral KV store is asking for trouble.

Re: LearnDB: Learn how to build a database

#23
post #5

How is this #2 on Hacker News? There's literally nothing here about how to actually build a database (yet?) Instead, there's just a key-value store implemented on top of a javascript hashmap and a filesystem.

I'd be very interested in a book or blog series similar to the scads of "how to build an os from scratch" or the raytracer projects, like Peter Shirley's little One Weekend series. Is there anything that scatches that itch?

Nand2Tetris is a fan-favorite: https://www.nand2tetris.org/

Build a working Tetris game from literal logic gates up.

Re: LearnDB: Learn how to build a database

#24
post #19

Earlier quoted context omitted.

JavaScript is extremely accessible and the warts can easily be worked around. I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t? Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anythin…

> and the warts can easily be worked around Not really. The author doesn't leverage any kind of static typesystem, which would at least mitigate some of the warts. He's chosen Node.js instead of leveraging the browser, so no free visualization layer for doing anything interesting with, unfortunately. > Have you worked with a JS language for any extended period of time? Most of my day job is working in a really, reall…

This was my thought exactly. For many high school and university students in the US, Java is their first language and introductory courses usually deal in unsatisfying toy applications. I could imagine a great primer into building a database that introduces all kinds of interesting computer science concepts as well as practical programming techniques (e.g. organizing a project as it grows).

Re: LearnDB: Learn how to build a database

#25
post #2

I realized how little i knew about how databases work until I watched this lecture series. https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA Does anyone have DB internal book recommendation that inst' boring as hell.

After hearing DDIA (https://dataintensive.net) recommended a few times, I picked it up and have been working through it recently. The book includes an incredible amount of references that provide further reading on individual areas.

Re: LearnDB: Learn how to build a database

#26
post #10

Why would a pedagogical project choose JavaScript of all languages? Unless the project specifically needed to leverage features of the language, or a web browser, it's an incredibly poor choice for building anything with well maintained abstractions. Or anything at all ready, when the language is covered in warts. I imagine the author hasn't yet discovered for themselves why it's a poor choice, given only a key-value…

JavaScript is extremely accessible and the warts can easily be worked around. I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t? Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anythin…

How is JavaScript accessible? Because of websites like the linked article, which covers the basics, that lets you build things? Because Udemy, codeacademy and all the others will teach you how to google program using it?

Maybe you’re right, but this page doesn’t actually teach you databases, and most of those courses don’t actually teach you programming.

At least not efficiently.

I have worked with JS for a long time, and I don’t hate it, but it’s such a terrible language and environment that the most popular part of it is literally a strict syntactical superset of of it.

I don’t particularly like Typescript by the way, and I don’t think it’s really that useful, but you can’t deny that most people do.

I think 95% of all projects would be better of using something that wasn’t JavaScript for the whole stack, and I think that’s the reason so few things are build on node. I like graphql as much as you do, I also think Prisma is okish, but I’d rather use Django, Flask, Java Spring it .Net Core because they are so much more efficient in the long term.

Of course on the client side, all the innovations and all the talent lies with JS, and there is some advantages for using JS for your whole stack. Those advantages end at the DB though, at least in my opinion.

This isn’t a problem, you can use Prisma for Postgres, and there are decent drivers for mssql, but I’d never advise people to use nosql unless they had a very specific reason for doing so, and I can’t think of one.

Re: LearnDB: Learn how to build a database

#27
post #19

Earlier quoted context omitted.

JavaScript is extremely accessible and the warts can easily be worked around. I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t? Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anythin…

> and the warts can easily be worked around Not really. The author doesn't leverage any kind of static typesystem, which would at least mitigate some of the warts. He's chosen Node.js instead of leveraging the browser, so no free visualization layer for doing anything interesting with, unfortunately. > Have you worked with a JS language for any extended period of time? Most of my day job is working in a really, reall…

No stupid questions time, but given all languages, would an array based language like J be a decent choice for building a database from scratch?

I mean, tables seem like doubly indexed arrays.

In my mind it wouldn't be readable but it'd be maybe less lines of code than other languages?

I have never thought about building a database that takes into consideration CAP theorem and ACID compliance, sounds super tough. I'm usually happy enough when I learn something neat about Postgres.

I thought this would be neat but it's just a kv store in JavaScript.

Re: LearnDB: Learn how to build a database

#28

Earlier quoted context omitted.

JavaScript is extremely accessible and the warts can easily be worked around. I spent a lot of my life being a JavaScript hater and after working with it for four years I can definitely see how it is really a simple starting point for a lot of concepts. Sure it has warts. What doesn’t? Have you worked with a JS language for any extended period of time? What about that made you not even want to consider it for anythin…

How is JavaScript accessible? Because of websites like the linked article, which covers the basics, that lets you build things? Because Udemy, codeacademy and all the others will teach you how to google program using it? Maybe you’re right, but this page doesn’t actually teach you databases, and most of those courses don’t actually teach you programming. At least not efficiently. I have worked with JS for a long time…

js is perfect for the resource constrained student of today ie their primary computing device is a mobile.mobiles ship with a js environment built in regardless of platform. i used to be like you until i had to start advising kids from low income environs in india on what language to learn(from a personal desire to create hackers with low resource access and seeing what happened+give them access to coding as a skill).

js really is the most accessible

Re: LearnDB: Learn how to build a database

#29
post #28

Earlier quoted context omitted.

How is JavaScript accessible? Because of websites like the linked article, which covers the basics, that lets you build things? Because Udemy, codeacademy and all the others will teach you how to google program using it? Maybe you’re right, but this page doesn’t actually teach you databases, and most of those courses don’t actually teach you programming. At least not efficiently. I have worked with JS for a long time…

js is perfect for the resource constrained student of today ie their primary computing device is a mobile.mobiles ship with a js environment built in regardless of platform. i used to be like you until i had to start advising kids from low income environs in india on what language to learn(from a personal desire to create hackers with low resource access and seeing what happened+give them access to coding as a skill)…

[deleted]

Re: LearnDB: Learn how to build a database

#30
post #4

For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…

On the subject of readable database source code: * LMDB is legendary for its performance, but it is source code is kind of hard to read. Found there's a pretty decent port to Java which looks a lot more readable [1]. * LMDB doesn't support SQL. But there's this SQL implementation: Apache Calcite [2]. Creating a DB using llmdbjava + Calcite sound like an interesting project. 1: https://github.com/lmdbjava/lmdbjava/tre…

Maybe it would be better to start with https://github.com/leo-yuriev/libmdbx
Post reply on HN