Hacker News .hnnew | past | comments | ask | show | jobs | submitlogin

Indeed, there's a real tension there between treating test as "real" code and letting them be separate. You end up getting bitten both ways, and we've gotten hit by all of them. Things can't all work the same, since if TestA and TestB share a common setup routine or helper and TestC comes along and needs to tweak that, you don't want it to hose TestA and TestB. On the other hand, if you have 1 production implementation of some interface and 50 different test mocks, you're in huge trouble if you need to change that interface, and life will be better if the tests either use the real implementation or all share a single mock (though even then there's a danger in the mock diverging from the main interface).

There's a similar problem with tests using their own code paths to set things up instead of the normal system path. For example, suppose you're testing a method that takes a User object. In your production system, users might only be created in a certain way, they might have certain fields required, they might have sub-objects attached to them, the database might enforce nullability or fk constraints, there might be insert callbacks in the code or in the DB, etc. If you try to just hack up a User in memory and pass it to the test method outside of the normal creation path, your test User object might differ in important ways from your actual User objects in production. So changes in the production code, for example assuming a field isn't null or relying on a default value, might cause the tests to break erroneously even though the app is fine. Then you have to find all the places you create Users in tests and fix them, or try to centralize things so you only have to fix them in one place.

Some of those problems can be partially avoided by proper decomposition and decoupling of the code, though oftentimes you have to have the foresight to do that before the tests get out of hand (and having the tests get out of hand is a good canary in the coal mine that your code is in trouble).

We actually went all the way to one extreme whereby we run our tests in the production environment as much as we can and avoid mocks and stubs; we're a Java servlet app (kinda the only way to do enterprise software these days, unfortunately), so we start up a Jetty server, an H2 database in memory, and go through the normal server-startup sequence before running most unit tests, which at least eliminates the test setup problem and a lot of the test/production divergence. It comes at a huge cost in terms of local test execution times, unfortunately.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: