One case I have trouble doing in Mathematica, etc. are variable transformations to increase numerical stability. A lot of times if you're clever you can do a variable change on an improper integral with (integrable) singularities to get an integral of a continuous, bounded, function over (0, 1). Working out this change of variables is something I find easier to do by hand.

I also generally have trouble getting Mathematica to do all of the work I need to do to come up with error bounds on numerical schemes.