I think that cursor should be used more with realtime data and more stable data like products in an e-commerce shop should use paginated queries. There's many UX benefits for paginated queries. It gives customer a clear overview of how many products there are and customer can also navigate faster to get a better understanding of what general prices are. If the table/list doesn't have good filters, customer can easily…
API pagination design
101–110 of 150 posts
Re: API pagination design
#102> One deficiency you can see is that it’s impossible to generate a “previous page” link It is perfectly possible to have a "previous page" link. Your pagination needs to support "before" cursor semantic, allowing a consumer to retrieve the N items before the row identified by the cursor.
It is also possible to create a "next page". You only need to take your current page's last id and use it in your query like "where id > current_last_element_id".
"first page" and "last page" are possible as well, simply do "where id > 0" for first and "where id You may need to invert order here and there, but it is perfectly possible to use cursor base pagination in a GUI as long as you don't need to provide jumping to a specific page number.
Re: API pagination design
#103Earlier quoted context omitted.
Though you can't avoid seeing an item just before it is deleted and skipping an item just before it is created. This isn't a huge problem but does mean that the list you end up with may not be a coherent view of the database. But if you require guarantees like that then you might be better of just using SQL directly. Or just include everything in a single response, HTTPS was designed to handle arbitrary length payloa…
If you care about that kind of guarantee you might want to keep a connection open and stream new updates.
You don't need that. You just need to get the collection resource to be cacheable and subsequently track your resources' state with conditional requests.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditiona...
Re: API pagination design
#104I think that cursor should be used more with realtime data and more stable data like products in an e-commerce shop should use paginated queries. There's many UX benefits for paginated queries. It gives customer a clear overview of how many products there are and customer can also navigate faster to get a better understanding of what general prices are. If the table/list doesn't have good filters, customer can easily…
That said, I think the percent of average ability internet users (ie. people visiting an ecommerce shop) will virtually never attempt to navigate by changing values in a query string param. High scores on a gaming forum, maybe more likely. Either way, if users need the ability to jump through pages, I would hope that would be exposed in a UI rather than expecting people to edit the URL directly.
Re: API pagination design
#105The Use The Index Luke material about this is essential background reading, but in terms of actually implementing: My favorite SQL implementation of cursor-based keyset pagination can be found on this Hasura issue: https://github.com/hasura/graphql-engine/issues/141 , specifically this comment: https://github.com/hasura/graphql-engine/issues/141#issuecom... . Something so elegant about this SQL. I enjoy it a lot. Als…
I implemented a keyset paging library for sqlalchemy which supports this, and found the nicest way to do it is to swap the comparisons for the descending columns. For instance if this is an all-ascending paginated query:
select ... where row(c1, c2, c3) > row(11, 12, 13) order by c1, c2, c3
Then with c2 descending it would look like this: select ... where row(c1, 12, c3) > row(11, c2, 13) order by c1, c2 desc, c3Re: API pagination design
#106Earlier quoted context omitted.
How do you mean that it changes the proposition?
If you order by id as a tie breaker it's guaranteed that ever new inserted row has a higher id than the last id in the dataset. If the id is a GUID it's possible to add a new row between the paginated API calls with a GUID lower than that of the last received row. Or does the cursor prevent that the new row is skipped? What if the pagination is by order by id only?
If you want to order by creation time, then have a column with creation time. With resolution down to nanoseconds it'll be unique. And if not, well your query should be deterministic and "order by ctime,id".
But the problem is always possible. Doesn't matter what method you use. It applies to OFFSET … LIMIT, guid, name, or ID.
It's up to you to decide what you want the user experience to be if someone is on page 1, presses "next page", but before you click someone deleted the entire first page. And maybe there is no page 2 anymore.
These questions are application dependent, I'd say. And while I may have my opinions on the best way to handle it, it's an orthogonal question to what this article is talking about.
Edit: Oh, and there's GUID type 1, so "GUID" doesn't inherently mean "not sortable".
Re: API pagination design
#107Earlier quoted context omitted.
You use the sort key as the page selector. So if it's cities sorted by name, the query is "give me the first 20 cities with names after 'Constantinople'". If Constantinople gets the works, it doesn't matter, because there are still 20 cities whose names come after that lexicographically.
If it is updated, though (in this case, renamed to Istanbul), it can appear later once again, and newly inserted rows behind "Constantinople" will never appear. I mean, these problems happen if you use offsets too, and my point is that they don't matter. The most hardcore solution to those would be pre-computing pages but I doubt it makes sense to bring even more state to the search results.
Re: API pagination design
#108Re: API pagination design
#109I think that cursor should be used more with realtime data and more stable data like products in an e-commerce shop should use paginated queries. There's many UX benefits for paginated queries. It gives customer a clear overview of how many products there are and customer can also navigate faster to get a better understanding of what general prices are. If the table/list doesn't have good filters, customer can easily…
A while ago, github replaced the pagination on their commits page to be based on a git commit ID instead of a page number. After that change, it made it a lot more difficult to find the oldest commit in a large repo - frustrating. That said, I think the percent of average ability internet users (ie. people visiting an ecommerce shop) will virtually never attempt to navigate by changing values in a query string param.…
If you look at any e-commerce systems, ranging from WooCommerce to Shopify, Amazon and Walmart, I think they all use pagination. Some Amazon pages do have infinite scroll however
I have also been frustrated without having the ability to quickly go through commits... Or I think releases for that matter?
For instance, browsing through tags/release versions, which should be a very valid use-case in my opinion also uses cursor based navigation.
Re: API pagination design
#110Earlier quoted context omitted.
non backend dev here. could someone elaborate what exactly is so opaque about pagination? and what makes offsets less opaque? does this term have meaning I don't know about? and why does the DB need to wait for some number of synchronous reads?
"offset" can be passed transparently to db to retrieve a range of records while "cursor" is customarily implemented in the API layer. I think that's what "opaque" meant in GP's context. A starting read is needed to obtain the first cursor, hence the synchronous read.