What do you mean by "rule"? Depending on the exact circumstances, violating terms of service or ignoring robots.txt may not be a violation of criminal law or create any civil liability. In particular, scraping public data is generally legal under the CFAA regardless of robots.txt content.
That's a cool article to read. It explicitly wonders whether a robots.txt is enough to revoke authorization. And it seems like the DoJ does allow itself to consider the different blocking mechanisms used (including robots.txt) on whether to prosecute.
The DoJ is explicit in saying that something like a Cease and Desist is enough, so if for example the NYTs found OpenAI's bot then that would likely be prosecutable.
