Machine learning (and more specifically here, supervised learning) is about predicting a specific attribute of a new sample, based on the attributes of the samples that you've acquired. For example, if you have access to the database of the clients of a bank, containing their attributes such as their income, their age, their occupation and whether or not the bank accepted to give them a loan, you may want to create a system that based on this database, can predict whether a new person will get a loan or not.
It happens that having too much different features is not necessarily a good thing, in a phenomenom called curse of dimensionality.
Due to this, we are interested in trying to reduce the number of attributes our algorithm will process. There are two big categories of methods to do that: feature selection and feature extraction.
In feature selection, you try to select the attributes that are the "best" to predict your value. For example, computing the statistical correlation between the attributes and the value you want to predict, and choose those with the highest correlations.
In feature extraction, you create new attributes that are a linear combinations of the original attributes. PCA is a feature extraction algorithm.