Emacs users will recognize many of the readline defaults -- unsurprisingly, given both are GNU software.
I use "yank-last-arg" most often, which is M-. by default.
$ mkdir abcd
$ cd
On Zsh, I have M-, bound to "copy-earlier-word", which makes M-. M-, M-, cycle through the arguments to the previous command.
In reading the manpage for Bash, which doesn't have this function, I've just found "history-search-backward", which searches through history to find commands based on what you've already typed. That will come in useful!
$ bind '"\er":history-search-backward'
$ bind
(Note that M-r is bound to "revert-line" by default, but I doubt I'll use that.)
These functions are all listed, with explanations, in the Bash and Zsh manpages.
Edit: As I see from another comment, a binding for "history-search-backward" to the Page Up key is in /etc/inputrc on Debian and derivatives.