Columbia, NYU, and Cooper Union are excellent schools. If they were located anywhere but in the center of the world's media and financial system, they would define that place as an unparalleled academic center.
The problem is that Columbia is situated so far away from the other two, and from the "heart" of the city, that the university simply cannot have the same overarching influence on society that other schools that are better integrated with their surroundings do (like Harvard and MIT with Cambridge).
In addition, it's a matter of pure density- Cambridge and Boston, to a lesser extent, is seen as a college town because of the density of college students living in that area. In New York, college students compose such a tiny percentage of the overall population, and exert such a small influence on everyone else, that they are hardly given a second thought when determining the dominant themes in a city's ideology.
