One problem with this plan is that mirrors are not perfect and some non-negligible fraction of the incoming light is absorbed and/or scattered away. Even if you could build perfect mirrors, you would still likely need to place the whole setup under vacuum to prevent the light from being scattered by air.
You are right that regular mirrors absorb or scatter light energy. However, to get 100% reflection it should possible with a setup where the light is getting refracted or bounced off critical angle.
http://en.wikipedia.org/wiki/Total_internal_reflection
This is commmonly used in Fibre optics.
Its more of an engineering challenge to come up with a setup to view your past than a theoretical one!
The scattering problem is one that confused me, too. At a light-year or several dozen away, there's no way you could find and piece together all the photons that bounced off the moment you're looking for. Am I misunderstanding something?