If they open-sourced it, cheaters could fork it and completely neuter the anti-cheat functionality.

Security through obscurity isn't good security. The cheat writers won't be stopped just because they don't have the source. And with open source, the vulnerabilities it has can be found and fixed by everyone.

It's all about obscurity when dealing with this type of malware. They are advanced rootkits used by malicious users tampering with distributed systems that must trust the client to a high degree or they simply will not work. As soon as obscurity is used by either side it inevitably becomes the tool both sides are forced to use.

Aren't they going to do that even if it's not open source? (I mean, patch the binary)

You can detect binary patching. You can’t really detect patching if they have control over the whole source.

That doesn’t sound right.

Can you describe a method of detecting binary patching that wouldn’t also detect a changed fork being compiled?

They specifically said they were externally audited, in this case.