> > Forgetting to change the code that consumes the API means that things don't work, which is visible right away.
> Oh how do I wish this were the case!
It's certainly not always the case, but it's a whole lot more likely to be the case than for unchecked documentation.
> > Documentation that can break visibly when things change is better than static documentation.
> Agreed, but I would not call your average run-of-the-mill unit tests "documentation",
I think "is it documentation" is probably more of a spectrum than any particular threshold, and run-of-the-mill unit tests probably do fall on this spectrum though I'd probably agree that they're not particularly far along it (though that surely varies with the habits of those writing the tests).
> nor can all documentation can be programatically tested.
As a practical matter, that's certainly currently the case - tooling is not set up for testing documentation, and there are things we'd want to check that would be hard to check in any event. Theoretically also, there are certainly properties that can't be statically demonstrated. I'm not entirely convinced that there's nothing we're interested in that couldn't eventually be got at for the programs we care about, though it's certainly a possibility. Regardless, it seems an ideal worth pushing towards, and if tested documentation is interwoven with untestable documentation such that some conceptual locality is preserved it's less likely (though absolutely still possible, to be sure) that you'll forget to update the other when you're forced to update the one.
> Oh how do I wish this were the case!
It's certainly not always the case, but it's a whole lot more likely to be the case than for unchecked documentation.
> > Documentation that can break visibly when things change is better than static documentation.
> Agreed, but I would not call your average run-of-the-mill unit tests "documentation",
I think "is it documentation" is probably more of a spectrum than any particular threshold, and run-of-the-mill unit tests probably do fall on this spectrum though I'd probably agree that they're not particularly far along it (though that surely varies with the habits of those writing the tests).
> nor can all documentation can be programatically tested.
As a practical matter, that's certainly currently the case - tooling is not set up for testing documentation, and there are things we'd want to check that would be hard to check in any event. Theoretically also, there are certainly properties that can't be statically demonstrated. I'm not entirely convinced that there's nothing we're interested in that couldn't eventually be got at for the programs we care about, though it's certainly a possibility. Regardless, it seems an ideal worth pushing towards, and if tested documentation is interwoven with untestable documentation such that some conceptual locality is preserved it's less likely (though absolutely still possible, to be sure) that you'll forget to update the other when you're forced to update the one.