Testing is a vital component of any software development. To be effective, testing must occurs at multiple levels, focusing on individual software units or the entire end-to-end system. In this post, we’ll concentrate on the former and discuss the tools and techniques Proloquor.net uses for unit testing their Java code, particularly those integrated into the Eclipse environment.
Test Driven Development
Test Driven Development (TDD) is a software process that focus on short development spins, centered around automated and, ideally, continuous testing at multiple levels. Proloquor.net generally breaks this testing down into 3 unique phases.
- Unit Testing – Validation of individual software components, even those that can’t stand on their own as a complete set of functionality. Because of the number of units involved, it is typically automated in its entirety.
- Component Testing – Validation of an individual software component. A component is something that stands on its own, like a web page or executable binary. Component testing is often, but not always, automated.
- System Testing – End-to-end testing of the complete system. Testing at this level focuses more on usability and product effectiveness, rather than component or unit integrity. It can be automated only in specific situations.
When taken to its logical conclusion, TDD requires the software team to create an environment of Continuous Integration (CI). Here, the goal is to maintain a working implementation of the complete systems at all times, even if it fails to meet the majority of requirements. This is in contradiction to the classical waterfall development model where all the software is written before integration and testing begins.
Unit Testing Fundamentals
Unit testing follows a very simple rule, for every piece of code you write, write a companion piece of code to test it. For MyClass.java, there should be a MyClassTest.java to go with it. The goal of the test class is to exercise all the functionally of the target, including error and exceptional conditions. It must explore the limits of all boundary conditions the target class imposes on its use.
One of the challenges with unit testing however is that most classes are not designed to run in isolation like this. To be useful, we need a framework to help stand up an environment for these tests. While we’re at it, a facility to validate results and present findings would be useful too.
The JUnit Framework
JUnit is the standard unit test framework for Java. It follows the xUnit standard of unit test frameworks, as do many others such as NUnit (Visual Studio C# and VB.NET), CUnit (C), and CPPUnit (C++). JUnit defines:
- Test Cases – Individual tests
- Test Fixtures – Preconditions that setup a Test Case
- Test Suites – Sets of Test Cases that require a given Fixture
- Test Runners – Executable problems that launch the tests.
It also provides a library of assertions that allow you to compare the output of a test with an expected result, and formatters that generate summary reports about the tests.
To illustrate unit tests with the JUnit framework, we need a sample class to test. Our recent post on Combinatorics provides an example to work on.
The Combinatorics.java class currently contains a single static function to calculate factorials. In addition to calculating factorial values, it must check that the input is not negative. If it is, the method throws an exception.
Now let’s look at the corresponding CombinatoricsTest.java class.
This test suite contains two test cases, each exercising features of the factorial() method. JUnit makes heavy use of Java annotations. Any method in the test suite annotated with @Test is automatically run when the test runner runs. The first case, testFactorial(), simply calls the factorial() method several times, and asserts that the answer is correct. assertEquals() compares the expected outcome of the method under test with the actual outcome; if it matches, execution continues. If it doesn’t, the test case aborts at that point and reports the failure to the runner. There are many assertXxx() options; the one that compares double data types requires a third ‘tolerance’ parameter.
The factorial() method can only take non-negative numbers as input. If it is passed a negative number, it throws an IllegalArgumentException(). This can be tested with the testFactorialException() case. It’s annotation includes the exception that is expected to be thrown in the case. If it isn’t thrown, the case fails.
The final step is to create a runner executable for all tests. While the JUnit framework provides facilities for this, we can also run them from within the Eclipse IDE.
Running Unit Tests in the Eclipse IDE
We know that Eclipse is a powerful Integrated Development Environment (IDE). Among its many customization is a built-in JUnit test runner and result formatter. It comes with the base JEE version of Eclipse you’ve already downloaded. Since we haven’t discuss using Eclipse for Java coding yet, let’s start at the beginning.
Create Your Class
Begin with creating a new Project by clicking File/New/Other/Java/Java Project. Walk through the wizard, giving the project a name, and otherwise accepting all the defaults.
Next, it’s good practice to create a new package. Click on File/New/Other/Java/Package, or click on the Package icon (), and enter the name of your package, like net.prolquor.utils. This will set up the appropriate directory structure for your classes. Be sure to check the box to create a package-info.java file. You can use it to store some documentation about your package; we’ll see how it’s used when we discuss JavaDoc.
Now you can create your class, Combinatorics.java in this case, by clicking File/New/Other/Java/Class, or on the Class icon (). Fill out the package the class is in, the name of the class, and any inherited or interface classes. Also make sure the ‘Generate comments’ box is checked. Now build out the class in the editor.
If your class had a main() method, you could run it from within Eclipse by clicking the Run button (). Instead, our test class will provide the execution environment. You can even use the built-in debugger ().
Create Your Test Suite
Eclipse can help you with this too. Click-right on the target class name in the Package Explorer window, then select New/Other/JUnit/JUnit Test Case. Walk through the wizard, entering the name and package of the test class. I suggest picking a different class than the target, such as net.proloquor.utils.test. Also make sure the ‘Class under test:’ is set to your target class. The next page asks you which target method needs a test case. Sometimes, test cases can cover multiple target methods. Once finished, you can build out your test cases.
Configure Your Test Fixtures
Most classes are quite a bit more complex than Combinatorics.java. In most cases, there is some amount of configuration that must happen before the test can be run, and possibly some cleanup afterwards. JUnit provides fixtures for this with additional annotations that identify methods that should be run before each test, as well as the entire suite.
Run Your Tests
Here’s the coolest part. Normally, we’d have to create a test runner application. Instead, we can use Eclipse’s built-in runner. Click-right on the test suite class and select Run As/JUnit Test. Eclipse runs all the tests and conveniently summarizes the results in a separate JUnit window.
Measuring Code Coverage
So we can now run our unit tests at the click of a button and see the results right in our IDE. But how good are our tests? That’s a difficult question to answer. One measurement is code coverage. Our goal here is to insure that every line of our target class is executed by our test cases. 100% code coverage may not be sufficient to insure effective tests, but it is certainly necessary.
Measuring code coverage can be done automatically by using runtime extensions like EMMA. EclEmma is an Eclipse plugin wrapper for EMMA that allows you to run code coverage analysis and view the results from within the IDE.
You can install EclEmma from the Eclipse Marketplace. From Eclipse, click Help/Eclipse Marketplace… and search for EclEmma.
Once installed, you can run any program from within EclEmma by using its own launch button (). The tests run as usual, but a separate ‘coverage’ window is available that summarizes the coverage of each file. The file contents are also color coded to shows what lines ran.
Here we see our tests do pretty well, covering 90% of the target class. Only one line, the initial definition, was not run because no test case created an instance of the class. We could create an additional test that does this to reach 100%.
We covered a log of ground today. If we’re disciplined about unit testing, our code will be much more reliable and we’ll spend a lot less time debugging it. Next we’ll want to cover documenting our code with JavaDoc and setup a continuous integration server with Jenkins.