Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We switched to cursor-based pagination

moderntreasury.com

Post reply on HN
Print view View original

21–30 of 118 posts

Re: We switched to cursor-based pagination

#21

Post by dinkledunk »

jsmith99 wrote: post #19
dinkledunk wrote: post #12

Earlier quoted context omitted.

how to jump to an arbitrary page?

Do you really need to jump to an arbitrary page and land on the exact item? For many applications an approximate jump is fine. If your column is fairly uniformly distributed you can guess the index for any arbitrary page.

Yes, my business users will feel like they don't have sufficient access to their data if they can't.

> If your column is fairly uniformly distributed you can guess the index for any arbitrary page.

I don't think that'll work in a multi-tenancy situation with complex filters.

Re: We switched to cursor-based pagination

#22

Post by imbusy111 »

drdec wrote: post #20

I was hoping to read about how they handled cleaning up cursors, what the effect on the server was when having a bunch of open, long-running cursors, etc. Unfortunately the article only treated the subject at a superficial level. So, anyone here implement pagination via cursors? What do you find to be the drawbacks and how do you mitigate them?

You probably assume they are talking about database cursors. The cursor is just a record ID in this article. There is no long term storage of database cursors on the server. Assuming you can sort all your data, next query just returns all records after the given record ID, plus the record with that ID.

One corner case would be if the cursor record is deleted. I don't see it mentioned how they handle it.

Re: We switched to cursor-based pagination

#23

Post by erikpukinskis »

drdec wrote: post #20

I was hoping to read about how they handled cleaning up cursors, what the effect on the server was when having a bunch of open, long-running cursors, etc. Unfortunately the article only treated the subject at a superficial level. So, anyone here implement pagination via cursors? What do you find to be the drawbacks and how do you mitigate them?

Yah, another downside of cursor-based pagination is: what happens when the record the cursor refers to is deleted?

Do you just crash and ask the user to start over?

Do you have to nudge open cursors on every delete?

Re: We switched to cursor-based pagination

#24

Post by jsmith99 »

G3rn0ti wrote: post #3
mike_hock wrote: post #2

TL;DR: The headline. TFA doesn't really add any information.

Well, except that TFA explains what DB cursors are and why they are faster than page offsets (because they skip DB entries).

I'm pretty sure they aren't using actual SQL cursors as that wouldn't scale well so it's probably not a 'DB cursor'. It's a shame as actual cursors are the native database way to do pagination.

Re: We switched to cursor-based pagination

#25

Post by christophilus »

drdec wrote: post #20

I was hoping to read about how they handled cleaning up cursors, what the effect on the server was when having a bunch of open, long-running cursors, etc. Unfortunately the article only treated the subject at a superficial level. So, anyone here implement pagination via cursors? What do you find to be the drawbacks and how do you mitigate them?

It’s not a cursor as in a relational database cursor. It’s a cursor, as in an ID of an item in a stably ordered set. There’s no long-running anything to be worried about.

Re: We switched to cursor-based pagination

#26

Post by erikpukinskis »

Worth noting that another solution to the problems with offset cursors is to do updates. When you add/remove rows you just update the results set and then there’s no issue with missing or duplicated rows.

Not easy to do of course, but it’s one direction you can go.

Re: We switched to cursor-based pagination

#27

Post by int0x2e »

fein wrote: post #18

I have a few main requirements for what I consider easy to use pagination. 1. I can set how many items per page. 2. I can get to an arbitrary page either through a path/ query param or at least close with a pagination row that contains a way to jump around. If an item gets removed, whatever I was looking for should still be in the same vicinity. 3. As a result of #1 and #2, I can go back and find items I saw previous…

I totally agree with you from a usability standpoint, and while it is possible to make this work well, for larger-scale services it is rarely without costs, and for the most part - people don't seem to care as much as you'd think.

While it can be frustrating, I doubt much revenue (relatively speaking) was lost due to this issue, which means for most apps and services, it will likely not get done.

(I'm not disputing the merit of your arguments, just explaining why this will rarely get done well in real life...)

Re: We switched to cursor-based pagination

#28

Post by [deleted] »

imbusy111 wrote: post #22
drdec wrote: post #20

I was hoping to read about how they handled cleaning up cursors, what the effect on the server was when having a bunch of open, long-running cursors, etc. Unfortunately the article only treated the subject at a superficial level. So, anyone here implement pagination via cursors? What do you find to be the drawbacks and how do you mitigate them?

You probably assume they are talking about database cursors. The cursor is just a record ID in this article. There is no long term storage of database cursors on the server. Assuming you can sort all your data, next query just returns all records after the given record ID, plus the record with that ID. One corner case would be if the cursor record is deleted. I don't see it mentioned how they handle it.

[deleted]

Re: We switched to cursor-based pagination

#29

Post by thaumasiotes »

G3rn0ti wrote: post #3
mike_hock wrote: post #2

TL;DR: The headline. TFA doesn't really add any information.

Well, except that TFA explains what DB cursors are and why they are faster than page offsets (because they skip DB entries).

> TFA explains what DB cursors are and why they are faster than page offsets (because they skip DB entries)

Except that no such information appears in the article. Here's the entirety of the explanation:

>> Once you’ve been returned a cursor, you can provide it in your requests to act as a farther-along starting point within your data entries. The server is then able to efficiently skip all entries that come before your specified cursor value

Re: We switched to cursor-based pagination

#30

Post by hamandcheese »

erikpukinskis wrote: post #23
drdec wrote: post #20

I was hoping to read about how they handled cleaning up cursors, what the effect on the server was when having a bunch of open, long-running cursors, etc. Unfortunately the article only treated the subject at a superficial level. So, anyone here implement pagination via cursors? What do you find to be the drawbacks and how do you mitigate them?

Yah, another downside of cursor-based pagination is: what happens when the record the cursor refers to is deleted? Do you just crash and ask the user to start over? Do you have to nudge open cursors on every delete?

Instead of the cursor being an ID, it could directly encode whatever column(s) you are sorting by. Then you don’t have to locate any record in particular, you can always return records that sort after the cursor.
Post reply on HN