IIRC, Bernstein actually disagrees with (almost) any use of rdrand. Consider, as a toy example, an rdrand that poisons a register, so that xor'ing it with anything results in a known value; this might be worse than useless.
I also don't see why we should assume that Intel would implement RDRAND as looking at the randomness pool and at the same time believe that they can't do even more funny things when they control the whole CPU. If the active maliciousness is assumed then the CPU can't be used at all.
I feel like I remember him saying something about how as long as it's just another thing hashed into the randomness pool, it's not that big of a deal. But I haven't seen that in writing anywhere. I could just be wrong.
(http://blog.cr.yp.to/20140205-entropy.html: "imagine RDRAND microcode that's looking at the randomness pool that it's about to be hashed into")
I'm not sure I agree with Bernstein here.