Live data from Hacker News

How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

youtube.com

101–110 of 134 posts

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#101
post #96

Earlier quoted context omitted.

I imagine you've written more than your fair share of PROGRESS 4GL code in the past .. your qualifications questions are pretty much straight out of the PROGRESS 4GL user guide .. ;)

Nope! I've never used postgres at all. Most of my RDBMS work was done on HN NSFW ALERT - oracle :-) But your comment supports my general point, which is, that data modelling and relational schema design skills are product agnostic; normalization is normalization, as it were.

Just to point out, "PROGRESS 4GL" is a different thing, not PostgreSQL based.

It seems to be called OpenEdge Advanced Business Language these days:

https://en.wikipedia.org/wiki/OpenEdge_Advanced_Business_Lan...

And yeah, the terms you mentioned are foundational RDBMS terms, common to all reasonable implementations of RDBMS's. :)

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#102

Earlier quoted context omitted.

>Databases are not very complex and use pretty much only textbook data structures and algorithms skeptical expression

"________ is not very complex and uses pretty much only textbook data structures and algorithms" is pretty much a false statement for any engineered product.

And the difference is usually close to nil for understanding them. (Also see squirrelicus' comment). Whether my database uses a B-tree or a proprietary, patented, insanely complex and finely tuned data structure that does essentially the same thing as a B-tree just a little bit faster is, for all intents and purposes, not a relevant distinction.

Now it's true that most databases have a ton of features and even more optional features that can (and will) interact in interesting ways, but I thought it was fairly obvious that most applications use very few or none of these. They are the 90/10-sort of features; 10 % of a database vendor's customers need 90 % of the specialized features in a database, and every single one of them uses a different handful.

Obviously you don't need to understand all these specialized features to use a database; you only need to grasp the handful if any at all you actually need at a time. Applications striving for wide database compatibility tend to rarely use any of these, simply because they don't exist in all databases, or work differently, or have divergent interfaces.

So any time you have an application that runs on MySQL or postgres in production but is developed and tested on SQLite (an antipattern itself, but I digress), you can be assured that you'll only see fairly basic DDL and SQL.

(You also seem to be intermingling understanding and building. I can use and understand how a typewriter works without having a clue how to build one. Yes, there are lots of hard problems solved by databases, but how they do it is mostly a don't care. I don't have to care how SQLite does power-fail-safe transaction, it does and what that means for me, is all I have to know as a user.)

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#103

Earlier quoted context omitted.

> what if there was a different query language with explicit index syntax.. There is, it's a feature in MySQL called Index Hints [1]. [1] https://dev.mysql.com/doc/refman/8.0/en/index-hints.html

Or oracle has it A DBA can even sit there as queries fly past, and add hints on the fly. And then you change a query from "select" (lowercase) to "SELECT" (uppercase), and query plans break and you break production. Fun times

Could you elaborate why changing select from lowercase to uppercase would break anything?

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#104
post #44

Earlier quoted context omitted.

The performance characteristics of them, though, are easy to learn, and easier to get a grasp on via EXPLAIN and such.

Given a database and a query, yes, you can understand the execution profile. What you can't do is start out with a schema and a query and understand the execution profile. Query planners take statistics into account when determining which indexes to use or to fall back to scanning, and can normally only examine a fraction of possible join orders. So you can usually only fully predict the performance profile of simple…

I might add, those processes have little to do with SQL. When you have a lot of data, and a lot of queries, then you're going to have to monitor and optimize your databases.

I might also add that a basic understanding of data modelling and what an index can and can't do is sufficient to avoid many, many performance pitfalls (this again has mostly nothing to do with SQL per se). Any undergrad course on databases teaches these basics.

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#105
If you are a Java programmer and want to learn how an SQL database engine works, take a look at the source code of H2.

Even better, try to add a basic feature to H2 (eg. a new built-in function). It is surprisingly easy, and you come away with a decent understanding of the basics of building an SQL database engine.

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#106

Love how he started. People who don't stop their conversations for a presenter are the worst. People who don't stop their conversations for a presentation by Richard Hipp deserve a spell of laryngitis.

Spell of laryngitis for Darwin awards winner is the new prize.

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#107
post #95
post #77

Earlier quoted context omitted.

SQL just wasn't designed for performance. That is strictly true in the literal sense that SQL is just a textual representation of relational algebra and calculus, and noone says a mathematical notation is "designed for performance" or otherwise. But in a more practical, useful sense, it's the language most designed for performance, since the query planner has so much leeway to perform optimisation. It can do more dra…

If you cannot predict performance of a query you cannot write a query with guaranteed performance characteristics and cannot guarantee performance of your application. So, no, SQL and RDBMSs in general are neither designed for performance nor are any good at it. Which is one of the reasons we have the whole world outside of them.

you cannot... you cannot... cannot

You keep using that word. I don’t think it means what you think it means.

Because people have been doing it since the 1980s...

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#108
post #36

Earlier quoted context omitted.

I completely disagree. Unless your data requirements are very specific, mem only, or not adapted to the relational paradigm any SQL engine will provide you with the best and more efficient algorithms to manipulate your data in the most common situations. I think that if any, databases should be used more.

> Unless your data requirements are very specific, mem only, or not adapted to the relational paradigm any SQL engine will provide you with the best and more efficient algorithms to manipulate your data in the most common situations. Too bad that Michael Stonebraker, Turing Award winner, disagrees with you. SQL are not the best solution for any common use case from the performance perspective. Nevermind what they do…

> IMHO less people should default to using a database upfront. At least while protyping the idea.

Surely it should be the opposite? While you're prototyping you should use a DB by default, then switch to your own implementation if you find out that it will speed things up (and you need that speed). It's not like the code that you would replace a DB with is going to be trivial, using a DB is going to keep the code simple until you need it to be complex.

Re: How SQL Database Engines Work, by the Creator of SQLite (2008) [video]

#110

I'm not going to call anyone out here, but why do people keep using the word orthogonal? It doesn't even compute. It doesn't even make any sense, in how they use it in relation to the topic. Are the issues at right angles of one another? No. Are the issues statistically independent of one another? Perhaps. I suggest to use a more appropriate descriptive word to describe the situation. You folks should read the urban…

> You folks should read the urban meaning of orthogonal, to understand how people roll their eyes at you, when you inappropriately use the term.

If that mattered at all, then we'd have stopped using other remapped words first, like "tree".

Post reply on HN