> If you build a $5 million dollar house, and a multimillionaire buys it as their fourth house, this does not reduce housing costs for anyone.

There is not an infinite number of people looking for second homes. Build enough $5 million dollar homes and you'll run out eventually.