I've used Octave and the Python stack and I can confirm that it is much easier to focus on the problem at hand using Octave. One can focus on the math and algorithms and not the language.

It is the same for me with R vs. the Python equivalents.

So your strategy of designing in Matlab and porting later (if necessary) makes a lot of sense.