Pex and Moles – Automated Whitebox Testing for .NET
Pex and Moles basics
Now some of you might be thinking what is Pex and how can a Mole help me test anything. Well Pex is a testing tool which helps you generate unit tests. And when i say Moles i am not thinking about the animal instead i am thinking about a framework which enables you to isolate parts which are tested from other application layers.
Pex is a tool which can help you generate inputs for your unit tests. To use Pex you have to be writing Parameterized Unit Tests. Parameterized Unit Tests are simple tests which accept parameters and Pex helps you generate these parameters automatically.
Because Pex explores the execution tree of your code, it also tries to enter inside all the mocking frameworks which you might use. This can be problematic, since Pex will generate inputs which will cause exceptions inside the mocking frameworks. By contrast Moles generates simple stubs of classes containing delegates for each method, which are completely customizable and transparent. Moles allows you to stub static classes, including the ones of .NET framework which are usually problematic to mock(typically DateTime, File, etc)
As it says on the official web: “Moles allows you to replace any .NET method by delegate”. So before writing your unit test, you can ask Moles to generate the needed stubs for any assembly (yours or other) and than use these moles in your tests.
How does Pex work
Pex enables parameterized unit testing, an extension of unit testing that reduces test maintenance costs. A parameterized unit test is simply a method that takes parameters, calls the code under test, and states assertions. Pex analyzes the code in the parameterized unit test together with the code-under-test, attempting to determine interesting test inputs that might exhibit program crashes and assertion violations. Pex learns the program behavior by monitoring execution traces, using a constraint solver to produce new test cases with different behavior.
This means that Pex can help you in understanding the input/output behavior of your code, finding inputs that cause the code-under-test to crash, and most importantly exploring parameterized unit tests to check whether your code implements the desired functionality for all inputs.
When you run Microsoft Pex on your .NET code, Pex generates test cases by analyzing the code-under-test. For every statement in the code, Pex will eventually try to create a test input that will reach that statement. Pex will do a case analysis for every conditional branch in the code—for example, if statements, assertions, and all operations that can throw exceptions.
The result appears as a table that shows each test attempted by Pex. Each line in the table shows the input and the resulting output or exceptions generated by the method for the specific input.
The result is a minimal test suite with maximum code coverage. Pex creates tests and mocks for MSTest. When a generated test fails, Pex often suggests a bug fix based on the Pex systematic program analysis. This is a fascinating feature that really helps you code better software that can handle inputs and outputs in a better way.
In my opinion Pex is a great tool when it comes to code coverage. It will exercise all the paths in your code to look for errors or exceptions. However sometimes you will have to generate the data for your test by hand and provide them to Pex. Moles is a great tool to provide stubs for static methods (and specially static framework’s methods) which normally are hard to test. It also cooperates well with Pex, because it is completely transparent. For each of you abstract classes or interfaces a stub is generated with delegates that you can redefine to fit your needs. If you would try to use other mock/stub framework, Pex will try to enter the scenes behind the framework, which might result in unexpected exceptions. However Moles lacks the “mocking” functionality. You can substitute any method with a delegate, but there is no build-in function which would tell you if the delegate was invoked. On the other hand this functionality can be easily developed.
Myself i have just started using Pex on a project because the current tests at work have very low coverage. It is my hope that Pex will help me increase the code coverage of the tests exponentially For more information about Pex and Moles head over to the Pex Microsoft Research page here.