Non-correlation between dietary cholesterol and blood cholesterol would most likely be due to the fact that blood cholesterol, and blood lipids generally, are more strongly related to total caloric intake.
The correlation with testosterone is equally invalid. Low-hanging fruit: it is not the simplest steroid (1). But lets go a bit further: by this logic, men with hypercholesterolemia should be bulls among men. Clinically, this is not the case (FWIW, I'm a physician and have diagnosed a few of these). Yes, cholesterol is a precursor to testosterone. But it is also a precursor to estrogens. Most cholesterol is synthesized by the cells themselves. There is no essential dietary minimum of cholesterol. You can't really avoid it in your diet either, because all cells have cholesterol (broccoli, beef, rice, you choose). Never mind the thermodynamic equilibria of the various enzymes involved, the hormonal regulation of the adrenals and gonads, etc, etc. And if you want to walk the biochemistry back further, the cholesterol is derived from lipids, which can easily be assembled from carbohydrates, especially in an anabolic state (i.e.: growing).
Counterpoints on your "males with..." theories. Spatial reasoning differences disappeared in the only known study of two genetically identical societies where one was matriarchal and one was patriarchal (2). From a more consequentialist perspective, vegetarianism increases with income in developed countries. (3)
The cholesterol in arterial plaques is a red herring. That cholesterol represents something less than a rounding error compared to total body cholesterol, and even less when compared to the total flow of cholesterol that must pass over the plaques in named vessels. A plaque is like a scab of the arterial wall. The plaques are more likely due to repetitive macroscopic injury potentiated by weak connective tissue, due to the connective tissue molecular injuries (collagen cross-linking, glycosylation, etc) caused by excess free radicals and other high-energy intermediaries (introduced from, e.g., smoking, excess dietary calories).
The answer remains the same: eat less. If your weight is outside the normal range according to wolfram alpha, you probably need to visit bwsimulator.niddk.nih.gov.
I assure you, many of your forefathers on the Serengeti lived long lives without jumbo eggs, and still had some wicked hacks (like wheels, music, property rights, etc).
People want to hack their bodies. Hack your relationship to the society you're in, that's the problem: figure out how to eat in moderation despite all the messaging. Get rid of Earl K. Butts' stain on the farm bill. Get rid of the farm bill entirely. Quibbling about tenth-degree issues like the relationship between eggs and testosterone is just a win for Monsanto. They got you to talk about something other than the problem. The problem is there's too much food.
(1) http://en.wikipedia.org/wiki/File:Steroidogenesis.svg
(2) http://www.pnas.org/content/early/2011/08/19/1015182108
(3) https://www.econstor.eu/dspace/bitstream/10419/50169/1/63222...