That doesn't change the calculus because the offset applies equally to both sides. Suppose you had paid the workers the same amount to build schools instead. They would still be able to buy houses with their salaries, but now there would also be a bunch of nice new schools.
I think the problem is we need to reach world stability. Look at Europe and Japan --economies which had invested relatively little into militaries, they are both reconsidering their previous position due to mainly two factors, the US disengaging, relatively speaking, and a perceived (and by perceived i don't mean imagined) threat from strengthening militaries in Russia and China, which given Europe and Japans previous dependence on the US left them in a era position to respond to growing threats.
This hits on honestly the only regret I have with the recent military history of the US. I don't mind that we spend $150 million on "warp speed death machines". I do mind that we spend some of that in the name of protecting countries which are perfectly capable of protecting themselves. Assisting allied forces with our own is fine; replacing them is not.
