Paper: among characteristics not mentioned,
where a passage occurs in a page (or how far within a book or article) is part of the context, and I suspect that human's spatial memory comes into play with this. Online infinite-scrolling content lacks this particular clue, and even PDFs only offer the "where on the page" vs. "how far into the book" aspect. I can also often remember my place in a book simply by staring at the page number and committing it to memory -- at least for a few hours or overnight. Though bookmarks are useful.
Electronic: Over half a century into the digital age and I think a significant problem is that we still haven't settled on the "right" format for real information consumption. There are plain-text documents, simple HTML, word-processing formats, postscript and PDF documents, and now eBook formats. Absent its lack of text search, ghostview -- 'gv' on Linux -- is still one of the best reading clients I know of, particularly in how it advances through the text. The Internet Archive's online book reader is the best I've seen bar none, in particular its light-weight feel and auto-cropping of text so that the actual content of the book is centered on the page. I've submitted a request to evince (the GNOME PDF reader) that it do similarly. Still, I find myself constantly fighting with even the best PDF readers at finding the right balance between text sized large enough to read and fiddling with positioning the page onscreen.
Electronic formats may not be as good for first encounter and deep studying, but they are immensely useful for quick reference: the ability to 1) carry around a library of thousands of volumes and 2) instantly search to or open to a particular page or phrase is hugely useful. The ability to create an external reference that will find the relevant passage in whatever form of the work you have handy (similar to a URL, but not specific to a given instance) would be magical.
Some limitations of electronic formats can be mitigated somewhat: there are bookmarks, annotations, and other forms of marking up documents, though none match the immediacy of a penciled margin note, underlined passage, or dog-eared page.