41–50 of 56 posts
Earlier quoted context omitted.
This looks super interesting! Although it still looks more complicated to understand than lmdbjava. I feel any C codebase will be more difficult to follow since there's gonna be a lot more code related to memory management that happens automatically in a GC language like java.
JVM and .NET ports - https://github.com/castortech/mdbxjni and https://github.com/wangjia184/mdbx.NET are listed there as well.
Earlier quoted context omitted.
I don't think the average case for new programmers looking to hack something is to start out building a solar powered aquafarm. It's better to teach the fundamentals first, which are easy enough to pick up from Javascript. Most programming courses start with something easy like Java, Python or Pascal for exactly this reason. Then later if they want to complete a more advanced project, they can pick up low level codin…
C/assembly are great starters because they teach you how computers actually work. Even a simple function in JS taking one variable abstracts a good deal of that away from you because it handles things like memory allocation for you. A lot of modern hopeful programmers that make it into our interviews can barely explain what the new keyword of a OOP language actually does, and way too many have no idea what a stack is…
The abstractions are what's important, memory layout is an implementation detail. Even when you're writing C or assembly, you're still thinking in terms of data flow and logic, you're just having to do a lot of the work manually. Learning the abstractions without the baggage of the implementation detail will get 90% of people 90% of the way. Once you have that solid foundation, you can go on to learn assembly or C if you need it because it's not that big a leap from a coding mindset to a hardware control mindset. But to go straight from no coding experience to hardware control is going to be more difficult for a new student to wrap their head around.
I realized how little i knew about how databases work until I watched this lecture series. https://www.youtube.com/channel/UCHnBsf2rH-K7pn09rb3qvkA Does anyone have DB internal book recommendation that inst' boring as hell.
https://www.apress.com/gb/book/9781484219638
It's specific to MS SQL Server but I found the chapters on log file management, indexes & isolation levels to be accessible and enjoyable.
Earlier quoted context omitted.
C/assembly are great starters because they teach you how computers actually work. Even a simple function in JS taking one variable abstracts a good deal of that away from you because it handles things like memory allocation for you. A lot of modern hopeful programmers that make it into our interviews can barely explain what the new keyword of a OOP language actually does, and way too many have no idea what a stack is…
for the vast majority of modern projects, you don't need the performance that manual memory management provides. I would bet that 90%+ of modern software is written in a memory-managed language. If you need to leap from JS to C, it's really not that hard. But trying to learn about stack frame allocation and RAII and cache line optimisation is a lot to take on when you've just learned what a variable is. The abstracti…
How can you really be tasked with solving problems if your first response is to look for a node package that does it for you? And that’s what JavaScript teaches you.
I’m not saying you shouldn’t use node packages. But learning to do that as your first steps into programming is robbing you of learning how to use code to solve problems.
Earlier quoted context omitted.
for the vast majority of modern projects, you don't need the performance that manual memory management provides. I would bet that 90%+ of modern software is written in a memory-managed language. If you need to leap from JS to C, it's really not that hard. But trying to learn about stack frame allocation and RAII and cache line optimisation is a lot to take on when you've just learned what a variable is. The abstracti…
I think you misunderstood me. It’s not the abstraction that’s important. It’s the problem solving that comes with knowing how the understanding. How can you really be tasked with solving problems if your first response is to look for a node package that does it for you? And that’s what JavaScript teaches you. I’m not saying you shouldn’t use node packages. But learning to do that as your first steps into programming…
That's not inherently an issue with Javascript. I agree that solving every issue with a library is bad in the same way that solving every issue by copy-pasting from stack overflow is bad, but it's not inherent to the language. I primarily work in Javascript these days and I've never actually seen a project that relies on an excess of micro-libraries. A good teacher will teach students to understand programming logic, not to lean on excessive crutches.
Earlier quoted context omitted.
> Fair warning: this is not for beginners. If you find the going too hard, start with a DB textbook. which one can teach how to build a DB system from scratch? I'm not talking about SQL theory or implementing a SQL parser but the actual persistence, indexing part.
Both of them, really. There's a lot to know if you want to build a modern database system. But no work I am aware of addresses the specific question of how to build a database system from scratch. A reasonable approach might be to start with this high-level paper, and follow papers they reference until they get specific enough to address your specific questions: Joseph M. Hellerstein, Michael Stonebraker, James Hamil…
For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…
Could you provide some examples of oddities?
It's been a very long time so I'm not sure I can name many specific examples, but IIRC some of the topics that became very clear were things like:
* the importance of the order in which JOINs and WHEREs are applied (to limit the number of rows being accessed per step)
* the relationship between columns that are selected and those that appear in WHERE and ORDER BY clauses.
* the value of tables with a small number of columns (limiting data that must be read per row)
* the cost/complexity of variable-width columns (VARCHAR vs a fixed-length string), again because of the time and complexity required to do something like "skip ahead three rows" in a data file
* the behavior of CLOB/BLOB types (stored external to the "main" table content)
* the various types of transaction isolation levels and the conditions in which it becomes hard or impossible to guarantee isolation without table or row locking
* etc.
Also, there's a post at http://heyrod.com/articles/radio-blog/pleasures-of-profiling... that describes a session of performance-tuning on the index implementation that gets into some of the nitty-gritty implementation topics. (But a lot of the issues addressed there were an artifact of Java's primitive vs. object representation, which has been reduced by things like generics.)
For complicated reasons I was involved in a successful project to develop an open-source, Oracle-SQL-compatible, transactional-integrity-preserving, extensible database designed for in-memory performance in Java. In the process of this development "suddenly" the reasons for a whole bunch of performance and syntax oddities in databases like Oracle and PostgreSQL became very clear. I was shocked to learn that was 15 ye…
Yup, a good way to understand why RDBMS are they way they are is to try to make your own in whatever language you choose. I've done this on a very elementary scale making a built in DB for a past project, at first it was single threaded, but when you start considering indexing and the datas structures necessary, it makes you understand and appreciate the man hours that have gone into projects like SQLite, PostgreSQL…
The more we used HSQL the more it became clear that it was more like a SQL-parser wrapping a simple key/value store than a full-on ACID database. We created AxionDB precisely because we needed the kinds of capabilities you mention and at the time HSQL did not provide them and wasn't remotely architected to support them.
To be honest though, creating a moderately robust RDBMS from "scratch" turned out not to be the most ambitious or complex part of the overarching project that spawned AxionDB. The harder part was trying to use Java's primitive, built-in HTML-renderer to create something approximating a fully featured browser. The effort and complexity behind something like Gecko, WebKit, Edge, Blink, etc. is very easy to underestimate. It's a hard problem, made much harder by having to tackle the kinds of content you find "in the wild. Frankly building a database was a much more straightforward problem than that.