The notion that new construction being 'high-end' won't improve the market on the low-end is erroneous in markets where the spread of rent budgets of people forms smooth, continuous gradient. But the Bay Area is not such a market -- nor are places where significant cash infusions such as very high pay at the high-end or subsidies like military housing allowances to people of middle incomes will distort the field upwards. Then there's Prop 13, which disincentivizes the sale of legacy homes, removing many residents and homes in the middle of the income/rent spread from the liquidity of the market.
Developers want to build units that are 'high-end' because it's the best way to recoup their costs. Urban construction is expensive and many projects are made possible by a combination of loans, private equity funding, and government subsidies. The increase of supply in high-end apartments creates competitive pressure on older units, places with less amenities, places with dated design, and places in inferior locations. Since housing sorting resembles an auction, people with the highest rent budgets have the most options. Landlords of less ideal places will see some tenants filter out and replaced. Eventually, the replacements stop coming, because all people of higher rent budgets have already moved to more desirable places. Instead of letting apartments sit empty, the landlord might lower the rent while still staying within the profitability threshold.
This doesn't work every time. If people of high rent budgets never stop coming, they never have to lower prices: there's an undersupply on the high-end too. This is clearly the case in the Bay and other desirable areas of California.