No. We're sure. We can observe that the universe was hotter and denser in the past because light takes time to get here, so when you look far enough into space you see older things. Also, there are tons of incidental facts about the doppler shift that can only be explained by Hubble Expansion. The best one that comes to mind is the "Finger of God" effect where galaxies are stretched out in doppler phase with an axis pointed toward the observer. This isn't caused by Hubble Expansion directly, but by the peculiar velocity (velocity within spacetime rather than metric expansion) caused by gravitation bonding acting against the Hubble Expansion. That is, when Hubble Expansion tries to pull a galaxy apart, the gravity of the stars keeps it together, and there are effects from that which we can calculate very precisely, and we see exactly what we would expect to.
The existence and relative long-term stability of Hubble Expansion is, frankly, one of the most certain things anyone knows about physics. If you can replace it with something else, it has to be something that reduces to the exact phenomenon of Hubble Expansion under the normal conditions our Universe is currently in.
I'm not quite sure I understand the effect with galaxies. So let's say from two opposite parts of a galaxy we would expect certain difference in red shifts because of the distance (one part is farther than the other), but what we observe is not that?
Suppose you could make a galaxy out of stars that have no mass. They're just big, bright balloons arranged in the pattern of a galaxy. If you looked at the balloon galaxy from far away, you'd expect that the part of the galaxy that's far from you would be red-shifted more than the part that's closer to you, due to Hubble Expansion. Also, as time goes on, you'd expect the balloons to drift away from each other, since the expansion causes everything to become farther away from everything else.
But because real galaxies are made of massive stars, they pull on each other and maintain the shape of the galaxy against the effect of the metric expansion. In addition to the doppler shift you see in the balloon galaxy, you get an added effect from the "peculiar velocity" of the stars perpetually falling toward each other within the expanding space time--but, of course, never actually getting any closer, since the effects cancel out. The parts that are closer to you have peculiar velocity away from you (because they're falling to the center of the galaxy, which is further away) and the parts that are farthest from you have peculiar velocity toward you (since the center of the galaxy is closer to you).
So, the red-shift from the distant parts is reduced by the blue-shift caused by their peculiar velocity in the direction of the observer, and the lesser red-shift of the closer parts is exacerbated by the peculiar velocity away from the direction of the observer. If we were to naively suppose that this "Finger of God" effect weren't happening, and that red-shift and distance behave as they normally do in accordance with Hubble's Law, we would be forced to conclude that all galaxies are somewhat "pancake-shaped" with every pancake facing toward Earth. This doesn't seem quite right, and fortunately the effect of peculiar velocity shows us that it doesn't have to be, and conveniently verifies the truth of Hubble Expansion.
Your question reads like you think there's an expansion from something to somewhere else. Like, as though people were periodically allowed to run in opposite directions from a 50 yard line, and we're observing them from the 35 yard line as we slowly run in one direction or the other. But that's not what's going on, given the near-uniformity of cosmic microwave background radiation.
The existence and relative long-term stability of Hubble Expansion is, frankly, one of the most certain things anyone knows about physics. If you can replace it with something else, it has to be something that reduces to the exact phenomenon of Hubble Expansion under the normal conditions our Universe is currently in.