Live data from Hacker News

LearnDB: Learn how to build a database

learndb.net

31–40 of 56 posts

Re: LearnDB: Learn how to build a database

#31
post #2

I realized how little i knew about how databases work until I watched this lecture series. https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA Does anyone have DB internal book recommendation that inst' boring as hell.

I found this one very good and easy to read:

https://www.amazon.co.uk/Database-Management-Systems-Raghu-R...

Re: LearnDB: Learn how to build a database

#32
post #30

Earlier quoted context omitted.

On the subject of readable database source code: * LMDB is legendary for its performance, but it is source code is kind of hard to read. Found there's a pretty decent port to Java which looks a lot more readable [1]. * LMDB doesn't support SQL. But there's this SQL implementation: Apache Calcite [2]. Creating a DB using llmdbjava + Calcite sound like an interesting project. 1: https://github.com/lmdbjava/lmdbjava/tre…

Maybe it would be better to start with https://github.com/leo-yuriev/libmdbx

This looks super interesting! Although it still looks more complicated to understand than lmdbjava. I feel any C codebase will be more difficult to follow since there's gonna be a lot more code related to memory management that happens automatically in a GC language like java.

Re: LearnDB: Learn how to build a database

#34
post #30

Earlier quoted context omitted.

Maybe it would be better to start with https://github.com/leo-yuriev/libmdbx

This looks super interesting! Although it still looks more complicated to understand than lmdbjava. I feel any C codebase will be more difficult to follow since there's gonna be a lot more code related to memory management that happens automatically in a GC language like java.

JVM and .NET ports - https://github.com/castortech/mdbxjni and https://github.com/wangjia184/mdbx.NET are listed there as well.

Re: LearnDB: Learn how to build a database

#35
post #28

Earlier quoted context omitted.

How is JavaScript accessible? Because of websites like the linked article, which covers the basics, that lets you build things? Because Udemy, codeacademy and all the others will teach you how to google program using it? Maybe you’re right, but this page doesn’t actually teach you databases, and most of those courses don’t actually teach you programming. At least not efficiently. I have worked with JS for a long time…

js is perfect for the resource constrained student of today ie their primary computing device is a mobile.mobiles ship with a js environment built in regardless of platform. i used to be like you until i had to start advising kids from low income environs in india on what language to learn(from a personal desire to create hackers with low resource access and seeing what happened+give them access to coding as a skill)…

Do they have internet? Because if they do, then cloud9 IDE is an excellent way to teach, any language too.

One of the best examples of this is CS50x from Harvard, which uses it (and other cloud services) to give students an IDE, automatic helpers, debuggers and code checkers.

Node is easy to set up and get going with, but if you’re teaching people how to hack, maybe it’s better to teach them something much more low level so they can actually build real things? I mean, if you want to build a solar powered aquafarm, or water pump, then C or Python and a raspberry gets you a lot further than JavaScript, and with C you actually learn how computers work.

But I guess it depends on the circumstances, and not knowing yours, then perhaps JS is a good choice.

Re: LearnDB: Learn how to build a database

#36
post #28

Earlier quoted context omitted.

js is perfect for the resource constrained student of today ie their primary computing device is a mobile.mobiles ship with a js environment built in regardless of platform. i used to be like you until i had to start advising kids from low income environs in india on what language to learn(from a personal desire to create hackers with low resource access and seeing what happened+give them access to coding as a skill)…

Do they have internet? Because if they do, then cloud9 IDE is an excellent way to teach, any language too. One of the best examples of this is CS50x from Harvard, which uses it (and other cloud services) to give students an IDE, automatic helpers, debuggers and code checkers. Node is easy to set up and get going with, but if you’re teaching people how to hack, maybe it’s better to teach them something much more low l…

I don't think the average case for new programmers looking to hack something is to start out building a solar powered aquafarm. It's better to teach the fundamentals first, which are easy enough to pick up from Javascript. Most programming courses start with something easy like Java, Python or Pascal for exactly this reason. Then later if they want to complete a more advanced project, they can pick up low level coding - it's not much of a leap, despite how fawny and melodramatic people get over coding in C/assembly/whatever.

Re: LearnDB: Learn how to build a database

#37
post #17

Earlier quoted context omitted.

> Why would a pedagogical project choose JavaScript of all languages? I don't think the JS part is the issue. Node.js does I/O, files and co. The issue is that the article doesn't teach how to build a database at all. It doesn't explain how to efficiently persist and fetch data from a file, indexing strategies with trees, concurrent file access,locking, basic transaction... that's what I expect from a tutorial about…

Plenty of languages do I/O, files, and co. Why choose a language plagued with warts and a half-baked ecosystem for something pedagogical? A language with all sorts of peculiarities from the '95 browser era, and not at least leverage browser technology? A language which gives you very non-interesting coarse control over resource, and has no concept of parallelism?

Easy access and fast results. Learning isn't about creating the best possible implementation. Python wpuld have been another suggestion from me personally. What language would you have used?

A lot of people without a formal educational background start with established scripting languages.

I you handle data, chances are that you will get in contact with javascript at some point.

Using coffeescript or typescript would be a worse choice for learning, even if they counter some disadvantages of the base language in question.

Re: LearnDB: Learn how to build a database

#38

Earlier quoted context omitted.

You might try this collection of papers about DB technology. http://www.redbook.io/ Fair warning: this is not for beginners. If you find the going too hard, start with a DB textbook. Also, that site doesn't seem to include the papers themselves. But most of those papers are very famous, so if you search for the titles, you should find copies. Worst case, you might need to visit a university library.

> Fair warning: this is not for beginners. If you find the going too hard, start with a DB textbook. which one can teach how to build a DB system from scratch? I'm not talking about SQL theory or implementing a SQL parser but the actual persistence, indexing part.

Both of them, really. There's a lot to know if you want to build a modern database system. But no work I am aware of addresses the specific question of how to build a database system from scratch.

A reasonable approach might be to start with this high-level paper, and follow papers they reference until they get specific enough to address your specific questions:

    Joseph M. Hellerstein, Michael Stonebraker, James Hamilton. 
    Architecture of a Database System. Foundations and Trends in Databases, 1, 2 (2007).
For the sort of low-level issues that you are interested in, it might be useful to study a well-regarded persistence package, such as Berkeley DB.

https://en.wikipedia.org/wiki/Berkeley_DB

Another system that might be worth studying is SQLite.

https://www.sqlite.org/index.html

Re: LearnDB: Learn how to build a database

#39
Not a lot of content yet, but building databases (even toys) is an incredibly interesting exercise which I encourage many more people to try out, so +1!

For anyone else who is interested in learning how to build a database, can I thoroughly recommend following along with Andy Pavlo's Advanced Database Systems course from CMU[1]. Every lecture is accompanied by reading lists, notes, and assignments. Whats more, I find Andy's style to be very easy to parse even on complex topics.

Even if you think you know a fair bit about this domain, you will likely learn a lot!

[1]https://www.youtube.com/playlist?list=PLSE8ODhjZXjYplQRUlrgQ...

Re: LearnDB: Learn how to build a database

#40

Earlier quoted context omitted.

Do they have internet? Because if they do, then cloud9 IDE is an excellent way to teach, any language too. One of the best examples of this is CS50x from Harvard, which uses it (and other cloud services) to give students an IDE, automatic helpers, debuggers and code checkers. Node is easy to set up and get going with, but if you’re teaching people how to hack, maybe it’s better to teach them something much more low l…

I don't think the average case for new programmers looking to hack something is to start out building a solar powered aquafarm. It's better to teach the fundamentals first, which are easy enough to pick up from Javascript. Most programming courses start with something easy like Java, Python or Pascal for exactly this reason. Then later if they want to complete a more advanced project, they can pick up low level codin…

C/assembly are great starters because they teach you how computers actually work.

Even a simple function in JS taking one variable abstracts a good deal of that away from you because it handles things like memory allocation for you.

A lot of modern hopeful programmers that make it into our interviews can barely explain what the new keyword of a OOP language actually does, and way too many have no idea what a stack is.

This isn’t useful when you write CRUD web-applications for a few thousands users, which is arguably a lot of modern programming is, but it’s extremely useful if you ever want to build something original.

Which is actually the key issue. You seem to think building a water pump isn’t a beginner project, but why isn’t it? It’s one of the most basic programs you could write. In fact it’s so basic that you could solve it mechanically, without the use of programming, if you have running water to power the timed open close mechanism of the pump and pull the water.

By contrast, a web site is infinitely more complex. Only you think it isn’t, because other programmers have build most of your tools for you. Which is great, you should stand on the shoulder if giants every time you can. What isn’t going to be great is when you’re tasked with solving a problem no one has solved for you first.

Post reply on HN