I made a little viewer to see the dataset. Spoilers, it shows you the answers. It's mainly to see the mistakes they've fixed on GitHub since it was released, and also to make proposing fixes easier.
Supposedly, they validated it upon release by showing each task to at most nine people and only keeping the ones that at least two people got correct in two tries. But still, they have had to subsequently fix more than a dozen of them.
https://9eorge.com/arc
Supposedly, they validated it upon release by showing each task to at most nine people and only keeping the ones that at least two people got correct in two tries. But still, they have had to subsequently fix more than a dozen of them.