Not imperfections, the fabric itself. If the balls were on a frictionless surface, their movement would be completely different, they would barely spin. Much of their movement is caused by their interaction with the fabric. But that is very hard to model from Newton's laws of motion.