They mention this risk in the Codex paper, too

> One challenge researchers should consider is that as capabilities improve, it may become increasingly difficult to guard against “automation bias.”