Sure, but if there are n interviewers, I get to waste n minutes of the company's time for each minute of my own wasted. For some panel interviews I've been in, this can be a significant multiplier that makes it worth it even if spite didn't already.