I recently did a bunch of "virtual onsite" interviews, mostly over zoom. I think the main frustration I experienced was the barely perceptible latency in the video chat. It really bites when you start talking at the same time as the interviewer and have to do the awkward, "oh sorry, no, go ahead", "I was just going to say..." conversation-resetting sidebar.

I felt like video quality was good enough to see non-verbal cues, and I think people subconsciously exaggerate those anyway when they're talking over a lossy channel.

On the other hand, it was a lot more comfortable to be in my apartment than in an unfamiliar office space.