What Is the Difference Between Supervised and Unsupervised Machine Learning?

Andrew Ng in his latest ML lectures details supervised and unsupervised Learning succinctly. Personally I haven't gone as far as I would like to further than that. Any advice on Where to start When learning ML.