Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pagination with rel=“next” and rel=“prev” (2011)

webmasters.googleblog.com

Post reply on HN
Print view View original

41–50 of 96 posts

Re: Pagination with rel=“next” and rel=“prev” (2011)

#41

Post by ldjb »

err4nt wrote: post #38
ldjb wrote: post #30

Earlier quoted context omitted.

I feel there are essentially two approaches. You could store the entire article text as a single chunk of data. The text would be marked-up to indicate where the page breaks are. Each time a page of the article is requested, the entire article is fetched from the database, but the website software extracts only the necessary extract. This is rather inefficient in principle, although it is possible to optimise, and it…

I'm no DB programmer, but despite the inefficiencies of the first approach (fetch entire article each time and display a different chunk) it would allow for users (or admins) to change page length after the article is written a LOT more easily than if each page was stored separately. Is this a potential concern of using the second method where each chunk is in its own field?

Assuming there is an actual user interface for composing and publishing articles, that interface doesn't have to be dictated by the way the data is stored.

Whichever method you choose, the software will need to be able to transform the data between two formats: one suitable for the user interface, and one suitable for the database.

However you designed the UI, it would be possible to use either approach for the database. For example, suppose the UI used a single textarea and had a special notation for delimiting pages. For the first approach, the contents of the textarea can simply be stored in its own field. For the second approach, the software would take the contents of the textarea, split it into its individual pages, then store each page separately.

If the UI had separate textareas for each page, then for the first approach, the software combines the contents of the textareas and puts in delimiters. For the second approach, each textarea is mapped to their own database field.

I suppose some of combinations are easier for the programmer than others, but so long as the UI is designed to be easy to use, the user need never know how the article is actually represented in the database.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#42

Post by kentt »

ThrustVectoring wrote: post #28
kmfrk wrote: post #4

Literally one of the greatest things about the Opera browser was that you could browse an entire forum or whatever (longform article etc) with the Space key, because the browser would automatically go to the `next` page on hitting the bottom of the page. They also did some cool stuff with swiping to the next page on mobile. Opera seemed like the only browser vendor that truly championed next/prev. I only do it now as…

The feature is available in some browser extensions as well. Vimium, for instance, has that available with ]]

Thank you. I use Vimium but never knew about that. I forgot how much I missed that feature on Opera.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#43

Post by xrstf »

kevindeasis wrote: post #19

How do you guys implement / What's the best practice for paginating your db using sql and nosql? I know there are a bunch of gotcha's to look out for

As fbonetti already said, LIMIT is the canonical way to do it with SQL databases (I would just add to their comment that you should ensure a positive page_size and offset, because "LIMIT -5" is invalid in at least MySQL, and to limit the page size to some sensible values).

For NoSQL, specifically (only?) CouchDB, you start iterating from a given key (in SQL, this would effectively be "WHERE id >= $start ORDER BY id ASC LIMIT n").

This difference is why I would recommend, especially for paginated REST APIs, to provide links to the next/prev pages, so API clients can follow these links without thinking about how pagination is accomplished exactly. To prevent people from guessing the inner workings and working around your links, you can encode the parameters in a "cursor" (so instead of ?pagesize=20&page=10 you could have ?cursor=base64encode("20,10") (SQL) or ?cursor=base64encode("$startKey") (NoSQL)). IIRC Facebook does it this way. Encrypting or authenticating the cursor is IMHO overblown here, given cursor values can be easily validated and rejected if fiddled with.

Using an opaque cursor also gives you the ability to change how pagination works without breaking existing API clients.

If you plan for lots of data and many pages, think twice before outputting links to all pages (on a website) or outputting the total number of elements (in an API). COUNT(*) on InnoDB is slow and I heard it's not the quickest thing in NoSQL databases either.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#44

Post by spottedquoll »

kevindeasis wrote: post #19

How do you guys implement / What's the best practice for paginating your db using sql and nosql? I know there are a bunch of gotcha's to look out for

Why would you break content into "pages"? If the requestor wants the data, give it to them

Forcing artificial page-breaks is abusive.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#45

Post by gnaritas »

spottedquoll wrote: post #44
kevindeasis wrote: post #19

How do you guys implement / What's the best practice for paginating your db using sql and nosql? I know there are a bunch of gotcha's to look out for

Why would you break content into "pages"? If the requestor wants the data, give it to them Forcing artificial page-breaks is abusive.

So if someone requires the entire table you think a single API call should just send it? Does it not occur to you that people used page data to limit response sizes and manage the performance of their API's?

Re: Pagination with rel=“next” and rel=“prev” (2011)

#46

Post by [deleted] »

toupeira wrote: post #36
kmfrk wrote: post #4

Literally one of the greatest things about the Opera browser was that you could browse an entire forum or whatever (longform article etc) with the Space key, because the browser would automatically go to the `next` page on hitting the bottom of the page. They also did some cool stuff with swiping to the next page on mobile. Opera seemed like the only browser vendor that truly championed next/prev. I only do it now as…

Which is why I love the Space Next addon for Firefox ( https://addons.mozilla.org/en-US/firefox/addon/space-next/ )

[deleted]

Re: Pagination with rel=“next” and rel=“prev” (2011)

#47

Post by kmfrk »

ThrustVectoring wrote: post #28
kmfrk wrote: post #4

Literally one of the greatest things about the Opera browser was that you could browse an entire forum or whatever (longform article etc) with the Space key, because the browser would automatically go to the `next` page on hitting the bottom of the page. They also did some cool stuff with swiping to the next page on mobile. Opera seemed like the only browser vendor that truly championed next/prev. I only do it now as…

The feature is available in some browser extensions as well. Vimium, for instance, has that available with ]]

This one also works admirably on Chrome by my brief impression: https://github.com/kenaniah/chrome-navigation-plugin.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#48

Post by austinjp »

amelius wrote: post #17
Roberto_ua wrote: post #8

What are the recommendations for infinite scroll?

Can anybody point to an implementation of infinite scroll that actually works? I know facebook's implementation doesn't work correctly because the scrollbar breaks when scrolling down using it.

Seems that we can learn lessons from comments here, and hopefully some can point to a decent implementation, or build one.

Salient points seem to be:

# Update URL during scrolling so links don't break, and link to the "deep" item not just the first page.

# Don't break history, so back/forward/reload buttons act as would be expected.

# Truly endless scrolling kills browsers due to consuming increasing resources, so don't do it. Seems that a compromise might be: larger page sizes that lazy-load content up to a sensible max length. For example, in a data set of 1,000 results perhaps display the first 10 immediately and lazy-load up to 50 during scrolling. Then normal paging occurs. (Max items per page could be user-selected, as is currently seen sometimes.) Items that have scrolled off the top of the page could be removed from the DOM too, and lazily reloaded on demand.

# Endless scrolling is not always appropriate, but there may be appropriate use cases. It would be good to identify these cases, perhaps through user testing.

# Obviously, it shouldn't break screen readers or text-only browsers etc. These should be handled gracefully.... which raises the question of how to effectively navigate through a large list, possibly of unknowable length, in these browsers.

I feel sure I've seen various of these, but not all together in one implementation. Anyone seen somewhere that does all this well?

Any further suggestions/criticisms of this pattern?

Re: Pagination with rel=“next” and rel=“prev” (2011)

#49

Post by detaro »

spottedquoll wrote: post #44
kevindeasis wrote: post #19

How do you guys implement / What's the best practice for paginating your db using sql and nosql? I know there are a bunch of gotcha's to look out for

Why would you break content into "pages"? If the requestor wants the data, give it to them Forcing artificial page-breaks is abusive.

So HN should be a single page with all submissions ever? ;)

Yes, don't unnecessarily break up content. But most pages are going to include lists of some sort that you want to be paginated in some way.

If you look at HNs /new queue, there are submission IDs in the next links, so you can click "more" multiple times without seeing content multiple times, even if new submissions have pushed them down in the global list by now. Useful if there are many updates and new entries come in at the "top" of the list.

Re: Pagination with rel=“next” and rel=“prev” (2011)

#50

Post by yeukhon »

xrstf wrote: post #43
kevindeasis wrote: post #19

How do you guys implement / What's the best practice for paginating your db using sql and nosql? I know there are a bunch of gotcha's to look out for

As fbonetti already said, LIMIT is the canonical way to do it with SQL databases (I would just add to their comment that you should ensure a positive page_size and offset, because "LIMIT -5" is invalid in at least MySQL, and to limit the page size to some sensible values). For NoSQL, specifically (only?) CouchDB, you start iterating from a given key (in SQL, this would effectively be "WHERE id >= $start ORDER BY id A…

How do you keep the cursor / know where you left off with the database? For example, I don't want to repeat what I saw in page 1 in page 2 because someone else just added more articles.
Post reply on HN