My guess is the framing in that paper was chosen to be provocative. The authors surely understand deep neural networks neither reason nor recite as we understand it in humans.

In any case, the conclusion was decidedly against reasoning.