We (Norway) did just that a few years ago; mostly, it was a success once the dust settled.
For instance, the coastal administration moved from the capital Oslo to Ålesund which is smack in the middle of a world-renowned maritime cluster; fisheries agency. moved to our #2 city (or #1 if you ask the locals) Bergen, historically the major export hub for the coastal fisheries, etc.
The key benefit is that it ensures there is a need for highly skilled labour also outside the capital; also, as you mention, it provides for more closeness to government - the sectors affected by the whims of a government agency has said agency within spitting distance, which helps.
For instance, the coastal administration moved from the capital Oslo to Ålesund which is smack in the middle of a world-renowned maritime cluster; fisheries agency. moved to our #2 city (or #1 if you ask the locals) Bergen, historically the major export hub for the coastal fisheries, etc.
The key benefit is that it ensures there is a need for highly skilled labour also outside the capital; also, as you mention, it provides for more closeness to government - the sectors affected by the whims of a government agency has said agency within spitting distance, which helps.