Live data from Hacker News

How does database indexing work? (2008)

stackoverflow.com

11–20 of 62 posts

Re: How does database indexing work? (2008)

#11

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

Unless your own benchmarks prove hash indexes to have major benefits for your queries, stick with the default B-tree index: you never know what queries you'll need in the future.

Re: How does database indexing work? (2008)

#12

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

Indeed, they perform much better, but caveat emptor: hash index operations are not currently WAL-logged. That means that: 1) if you crash during a hash index update, all bets are off and you'll have to rebuild, and 2) they will just be wrong in a replicated configuration.

EDIT: This is no longer true. Thanks latch!

Re: How does database indexing work? (2008)

#13

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

Less important now, but they're unsafe to use prior to PG 10.

As of 14.1, they still don't support unique constraints:

   select amname, pg_indexam_has_property(oid, 'can_unique') from pg_am

   amname | pg_indexam_has_property 
    --------+-------------------------
     heap   | ¤
     btree  | t
     hash   | f
     gist   | f
     gin    | f
     spgist | f
     brin   | f

Re: How does database indexing work? (2008)

#14

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

A btree is a kind of binary tree, one which is self-balancing to keep O(log n) lookups from turning into O(n) lookups which is possible without the self-balancing. (Think about a naïve binary tree where you insert elements in sorted order—then everything will be in the right (or left if you start with the maximum value) branch and you need to traverse the whole structure to find the node you're looking for.)

For sorting or adjacency queries, you want a btree index. A hash index can give you O(1) lookup with the drawback that you lose ordering completely (whereas getting a sorted selection of records with a tree is a O(1) operation). The trick here is having a hashing function which is both fast and avoids collisions (the worst case scenario is hash(x)=1 where everything hashes to the same value and you end up again with O(n) lookups).

Re: How does database indexing work? (2008)

#15

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

Indeed, they perform much better, but caveat emptor: hash index operations are not currently WAL-logged. That means that: 1) if you crash during a hash index update, all bets are off and you'll have to rebuild, and 2) they will just be wrong in a replicated configuration. EDIT: This is no longer true. Thanks latch!

This was fixed in PG 10 (Oct 2017). See https://rhaas.blogspot.com/2017/09/postgresqls-hash-indexes-...

Re: How does database indexing work? (2008)

#16
post #15

Earlier quoted context omitted.

Indeed, they perform much better, but caveat emptor: hash index operations are not currently WAL-logged. That means that: 1) if you crash during a hash index update, all bets are off and you'll have to rebuild, and 2) they will just be wrong in a replicated configuration. EDIT: This is no longer true. Thanks latch!

This was fixed in PG 10 (Oct 2017). See https://rhaas.blogspot.com/2017/09/postgresqls-hash-indexes-...

Oh, that's great news! (I last used evaluated them in August 2017)

Re: How does database indexing work? (2008)

#18
If you want to see a dumb example of how you can implement indexing on top of a SQL database without it, I wrote a tutorial on implementing basic indexes [0] as part of a series on making a SQL database from scratch.

And of course, Use the Index Luke is a great reference for the real world.

[0] https://notes.eatonphil.com/database-basics-indexes.html

[1] https://use-the-index-luke.com/

Re: How does database indexing work? (2008)

#19
My team has really enjoyed the concise and informative book [0] from Markus Winand. It comes up sometimes here on HN, and IMHO it's a great resource on the topic for anyone who touches a relational database in their line of work.

His website "Use the Index, Luke!" [1] is also great.

0: https://sql-performance-explained.com

1: https://use-the-index-luke.com

Re: How does database indexing work? (2008)

#20
post #14

I had assumed btree in Postgres was referring to a binary tree but that is definitely not the case. I also learned when to use a hash index, rather than a btree. (If I understand correctly if you’re only using the = operator rather than something that needs sorting with >, <, <=, etc. then a hash will perform better.)

A btree is a kind of binary tree, one which is self-balancing to keep O(log n) lookups from turning into O(n) lookups which is possible without the self-balancing. (Think about a naïve binary tree where you insert elements in sorted order—then everything will be in the right (or left if you start with the maximum value) branch and you need to traverse the whole structure to find the node you're looking for.) For sort…

> A btree is a kind of binary tree

By definition a binary tree means there are two children per node.

Binary is the one thing a b-tree is not.

Post reply on HN