What sort of embedded systems are you interested in? There are "embedded systems" which consist of standard server hardware with some preinstalled software (aka. "server appliances"); there are "embedded systems" which consist of low power boards running general purpose operating systems (e.g., cheap home routers); there are "embedded systems" which run specialized operating systems (e.g., the Martian rovers, running…
Right now I'm more interested in just getting started, so whatever would have a smaller learning curve in the beginning. I have had experience with installing custom firmware on home routers (dd-wrt) and that was what initially piqued my interest.
However, if you're after incremental steps on top of dd-wrt, set up a cross-compile toolchain for arm and start by just rebuilding the dd-wrt firmware you've already flashed. After you get that, make a "hello world" for the home router that blinks a light and outputs via serial. It'll give you a good feel for certain aspects of embedded development.
Have you tried to add USB to a home router? Certain models of router make this a relatively doable task, and there is usually at least a minimal bit of help (over doing it from scratch yourself), but you may already have most of the hardware to do it, possibly making it easier to start on.