I defer to Walter C. Willet, M.D., who writes in "Eat, Drink, and Be Healthy" (co-developed with the Harvard School of Public Health):
"There's no question that two types of fat -- saturated fat, the kind that's abundant in whole milk or red meat, and trans fats, which are found in many margarines and vegetable shortenings -- contribute to the artery-clogging process that leads to heart disease, stroke, and other problems...monounsaturated and polyunsaturated fats found in olive oil and other vegetable oils, nuts, whole grains, other plant products, and fish -- are good for your heart.... Our bottom line is this: It is perfectly fine to get more than 30 percent of your daily calories from fats as long as most of those fats are unsaturated."
"The term saturated means that the carbon atoms in a chain hold as many hydrogen atoms as they can...saturated fats come in gradations of bad...butter and other dairy products most strongly increase LDL (bad) cholesterol. Those in beef fat aren't quite as powerful at boosting LDL and those in chocolate and cocoa butter have an even smaller impact."
HDL and LDL are lipoproteins: "...fats must somehow get from your digestive system to your cells...like oil and water, fats and blood don't mix. If your intestines or liver simply dumped digested fats into your blood, they would congeal into unusable globs. Instead fat is packaged into protein-covered particles that mix easily with blood and flow with it. These tiny particles, called lipoproteins (lipid plus protein), contain some cholesterol to help stabilize the particles.
"Lipoproteins are generally classified by the balance of fat and protein they contain. Those with a little fat and a lot of protein are heavier and more dense than the lighter, fluffier, and less dense particles that are more fat than protein. The proteins also act like address labels that help the body route fat-filled particles to specific destinations.
"LDL is often referred to as the bad cholesterol...they can end up inside cells that line blood vessels. Once there, LDL is attacked by highly reactive free radicals and transformed into oxidized LDL. Oxidized LDL can damage the artery lining and kick off a cascade of reactions that clog the artery and set the scene for artery-blocking blood clots.
"In contrast, HDL particles sponge up excess cholesterol from the lining of blood vessels and elsewhere and carry it to the liver for disposal."
He goes on to list several studies: Ancil Keys' 1956 international survey called the Seven Countries Study which found a strong link between saturated fat and heart disease; the Framingham Heart Study, which identified high levels of cholesterol as linked to impending heart disease; the Nurses' Health Study and the Health Professionals Follow-Up Study (both very large cohort studies); the Lyon Diet Heart Study; and others. "In the 1950s and 1960s, dozens of carefully controlled feeding studies among small groups of volunteers showed conclusively that when saturated fat replaced carbohydrate in the diet, total cholesterol levels in the blood rose...."
The book also talks about the challenges of practical, large-scale studies of nutrition "in the wild": it's not easy to track and correlate people's eating habits over decades, for any number of reasons.