Risk aversion or maybe laziness.
I've had a similar line of thinking recently. Sure, average investors lose money on active trading. Well, let me tell you what an average investor looks like. They have little to no accounting background, no coherent thesis for their investments other than "it's hot", they lack even a rudimentary understanding of 101-level finance topics like NPV/IRR/cost of capital, they have no informational edge one gets from working in an industry, no understanding of tax law (either personal or corporate). They're basically gamblers. They look at stock charts, see a big line going up and to the right, and guess / let FOMO take the driver's seat. Oh, and they probably think watching Jim Cramer counts as "research". They also have no appreciation for the fact that individual securities within an asset class are highly correlated and misattribute the asset class's performance to their "stock-picking" prowess.
I've been thinking, if one is willing to do actual primary research -- reading financial statements, getting on investor calls, really thinking hard about long-term trends, reading history, etc., there's almost certainly alpha to be had. Do people honestly think indexing is the best way? The UK just mandated that all coal generation will be gone by 2025. Is this really a time to be buying coal stocks? Or cigarette companies?
For me it's about risk/reward. I was talking to a friend at a wedding two weekends ago who manages a prop book (his own cash) of about 2-3 million. If he can get 15%/year consistently, that's about 300-400k pretax. He lives in Puerto Rico where there's apparently a tax loophole where capital gains of any character -- short or long-term -- aren't taxed. I'd bet he's coming out ahead of 95% of the people reading this thread. And yet, life isn't all about money.
I guess what I'm saying is, I also disagree with this "orthodoxy" of index, index, index. Maybe I'll sound arrogant saying this, but I'm not an "Average" investor. I trade once/quarter, read the Ks and Qs for every company I'm investing in, and have written theses that guide my decisions. I'm not a millionaire yet but I'll let you guys know how it goes after a while. I was a pure, 10+ year indexer until about 6 months ago.