Why do we care so much about testing? After all, we write tests for homework assignments, and you aren’t likely to come back to a homework assignment in six months and add breaking features to it. However, we are in the business of teaching good habits and practices. There are several benefits of writing tests properly:
Unit tests help you determine code correctness. How else would you know if your assignment is ready to submit unless you had written a full unit test suite?
Unit tests provide a mental framework for code design. After writing out your test suite, you should have a better understanding of the flows of data within your program, as well as the outputs that your program should emit in response to certain inputs.
Unit tests are an easy way of letting other people jump into a codebase. While this does not apply to CIS 1210, in real life, code bases are shared between people. Not everyone can have a complete understanding of a software project; even Linus Torvalds, inventor of Linux, doesn’t fully understand the entire operating system and its ecosystem at this point. It is ostensible that someone can add a new feature or make a change to the code base that accidentally breaks existing functionality. A full test suite would catch this. What if an engineer came to work on a project years after you left that project? She wouldn’t be able to call you and ask you for your knowledge of the code base. Therefore, unit tests are therefore an artifact of your knowledge.
Unit testing saves your organization money. While this also does not apply to CIS 1210, this is probably the most important point, and definitely the most overlooked point. People write software to accomplish some business goal, whether that is creating a product, consulting for a client, performing academic research, or automating a manual process. If you write code that breaks something, you cost your organization money. For example, your customers might not be able to buy an item, you could lose a consulting client and damage your firm’s reputation, you could waste grant money, or you could perform some task incorrectly. Unit and integration tests are incorporated into all modern build systems; put simply, you will not be physically able to deploy code to production servers if any automated tests fail. Therefore, you are hedged against a large cause of performance regressions, and you save your organization money.
We encourage bottom-up testing in CIS 1210. Put simply, bottom-up testing means testing simple things before testing harder things. This is because of the compositional nature of software engineering: methods and abstractions build on top of each other. It is best to make sure that code at a lower level of abstraction is tested before code at a higher level of abstraction.
Why? Let us consider a motivating example. Imagine a 2D rendering engine in Java
that had a Point class, a Shape class, and a Scene class. A Shape is a
collection of Points, and a Scene is a collection of Shapes. You would
want to fully test the Point class first, then the Shape class, and then the
Scene class. This makes it much easier to determine the source of an error. If
some bug comes up in the rendering engine, you can be reasonably certain that
the issue is not with the Point class (the lowest of the abstractions), since
you already fully tested it! If we had done top-down testing, we would have
not known if the issue were in the Scene class, the Shape class, or the
Point class.
Bottom-up testing also applies to methods within a class. Imagine that you wrote a method that used three helper methods. Bottom-up testing mandates that you test the helper methods first, make sure that they work, and then test the larger method. The reasoning is similar as above.
Finally, bottom-up testing applies to the size of test cases for a specific method. You should order “small” test cases (edge cases and base cases) before large test cases. This way, if your code has a bug, you can reasonably determine characteristics of the bug.
Consider an example of bottom-up testing for the following specification that you would find in an interface:
/**
* Finds the maximum element of the input array.
*
* @param arr the array of integers to retrieve the max element of
* @return the maximum element in the array
* @throws IllegalArgumentException if the input array is empty or {@code null}
*/
int max(int[] arr);
The test cases you might want to consider are:
One should be reasonably convinced after passing this test suite that the max
function is correct. There is no magical answer that says that a size-four array
is the right one to stop testing at, but we can imagine that we have already
captured all of the method’s complexity in the above test cases.
At some point during this class, you are probably going to ask yourself or your
friends what the big deal with edge cases are. Why do we insist that you throw
so many IllegalArgumentExceptions and test all of those cases? There are two
answers: security and reliability. First of all, recognize that public
methods can take in any input that typechecks. We don’t have a choice but to
consider ugly inputs; public methods can be called by any client of our code.
There are edge cases that, if left unhandled, can return a stack trace or memory
dump to the user, leaking sensitive information about the internals of your
application. This can cost your organization money (not to mention cause a very
poor user experience). Second of all, if every possible input and output is
documented, a client of your code can write reliable code on top of your API.
While these concerns are not huge for the scope of CIS 1210, again, it is
important to build strong and correct habits.
As taught in CIS 1200, you should write unit tests before actually implementing code. This is useful because it allows you to get into a good workflow of adding code, running your test suite, and making fixes. It also encourages you to think more about the problem before solving it.
You can view the JUnit 4 documentation here, but the methods you are most often going to use are:
assertEquals(expected, actual): assert that two values are equalassertEquals(double expected, double actual, double delta): for floating
points in particular, you should specify an acceptable degree of error e.g.
0.001assertArrayEquals(_, _)assertTrue(_) and assertFalse(_)assertNull(_) and assertNotNull(_)You may notice that the expected value always comes first. While functionally the same, it is good style to always maintain this order! That way readers (and Eclipse) understand which value is actually correct.
It is oftentimes useful to re-initialize data structures before every unit test
is run, particularly if you are testing a method that mutates the data
structure. Imagine that you wanted to run dozens of unit tests over the same
graph, for example. You should use the @Before annotation to invoke a method
that is run before every unit test. For example:
private Graph g;
@Before
public void setUp() {
g = Graph.parseFromFile("directory.json");
}
There is a corresponding @After annotation that should be used for routine
clean-up. Since Java is garbage-collected, you generally would only need to use
this to close files, close network connections, etc. (You would also use this to
clean up your test database if running these tests as part of a deployment
infrastructure, but that’s beyond the scope of CIS 1210.)
Analogously, methods with the @BeforeClass annotation are run once (before
any unit test is run), and methods with the @AfterClass annotation are run
once (after the entire test suite is finished). You probably won’t use these.
It is easy to test for an exception to be thrown. Simply do the following:
@Test(expected = IllegalArgumentException.class)
public void testMaxNullArray() {
OurMathClass.max(null);
}