End-to-End Testing

In general, NUnit based unit test is used to test a single class or a single layer. We found this is very cumbersome and overkill in our development practice. Following are some problems we found with the unit test for a single class or a single layer:

1. There are a lot of small tests need to be created.
2. In order to test on a single class or single layer, you probably need to create many mock or stub classes to avoid intervention from other classes.
3. Come up with all different test scenarios are time consuming and difficult. In order to write test for a single class, you need to image a lot of ways to consume that single class, so you need to consider all possible situations though they are not realistic.
4. Refactoring could impact existing tests dramatically. The test is supposedly used to allow us can easily refactor our code without worry about breaking stuffs. However, when the test is too little and too focuses on single classes. It can be easily be broken by refactoring process. So, we need to spend significant amount of time to fix test or rethink scenarios.
5. Difficult to coordinate with QA team. QA team usually comes up test scenarios from user perspective. Because there is no direct relationship between their tests scenarios to our detail tests, so it’s almost impossible to directly use their test plan to create test.

Base on all the problems we found, we actually uses NUnit creating end-to-end tests. Use a common web application as example. A web application usually has UI layer, Application layer, and Database layer. The UI layer usually has a thin UI Rendering layer and a UI Model layer, The Application layer usually has a Domain layer and a Data Access layer.

There are two layers not NUnit testable, UI Rendering layer and Database layer. So, the rest layers we should have good enough tests to cover all possible situations. The end-to-end test we used will only create test from UI Model layer to Domain layer, so we skipped the un-testable UI Rendering Layer and we create a Mock Data Access Layer to avoid requiring a real database.

By doing this, we get following benefits:

1. No small tests all the places.
2. Can easily coordinate with QA team and even directly use their test plan as the base of our test.
3. Allow utilize refactoring to continue improve our design without worry about keep adding test.

By doing end-to-end test doesn’t mean we totally against test on single class or layer, we realize there are some situations require fine test case for single class or single layer to make sure the key functionality is completely covered.