Skip to main content

One Test Or Two?

Paul Grenyer & Chris O’Dell

"I know what you’re thinking: “Is that two tests or only one.” Well, to tell the truth, in all this agility, I’m not sure myself. But this is Test Driven Development, the most power development technique in the world, and could give you clean code, you’ve got to ask yourself one question: Do I feel expressive? Well, do ya, punk?"<

Paul: I have a Windows Presentation Foundation (WPF) application that loads a list of widget names from a database and uses them to populate a drop down box. There is no default widget, so the user must select one from the list and click the load button to load it. The behavior is such that before a widget name is selected the button is disabled and cannot be clicked.

One pattern often employed in WPF applications is Model-View-ViewModel. The view model can use variations of the Command Pattern for processing commands from and giving feedback, such as the enabled state of a button, to the View. MVVM is intended to give separation between a view and its logic. Therefore the view model is usually devoid of User Interface (UI) code and easy to instantiate and run unit tests against.

I have a unit test that instantiates a view model and checks that the load command cannot be fired until a widget name is selected. A simplified version is show below, without the view model instantiation:

public void testThatExecuteIsOnlyPossibleIfNameIsSet()
{
Assert.IsNull(model.Name);
Assert.IsFalse(model.CanExecute());
model.Name = "Something";
Assert.IsTrue(model.CanExecute());
}

Recently I have also been involved with the ACCU Mentored Developers project based around Growing Object Orientated Software Guided By Tests by Steve Freeman and Nat Pryce. In this book they write a lot about the naming and granularity of tests. So I posted the above code to the list and asked what the project members thought: “Should the above test be one test or two?”.

Chris: Without blinking I immediately replied that the above should definitely be split into two distinct tests so that a failure would be obvious from the test name. The above test has two asserts and as such either of these could fail.

Paul: Chris, and all the others who replied are of course right on the money (no one suggested it should only be one test), but something didn’t sit quite right with me. In this small example the extra code of two tests is not very much: the method definition, a pair of curly braces and some spacing. The problem for me is that test classes should not be treated that much differently to normal classes and any class with with a large number of methods becomes difficult to maintain and difficult to even to navigate. Although modern IDEs do make this easier with regions (C#) and code collapsing (Eclipse).

Chris: I firmly believe that it is worth the extra code - as many others also remarked, the five minutes now and extra curly braces could easily save twice that if you need to hunt down a related bug in future.

I went on to explain that in terms of application logic the two scenarios “with a name” and “without a name” are expected to be mutually exclusive and the two tests will ensure this by isolating each scenario and addressing them individually - with explicit test names.

In regards to the large classes this can be tackled by breaking your tests down into smaller classes, generally per scenario. For example, group all of the positive “happy path” tests together into one class and all the negative, error handling tests into another class and store both classes in a folder given the name of the class under test.

Paul: Again, Chris is of course right. Although the implication that a test method should only have a single assert is a whole other discussion.

Then Nat Pryce came along with another suggestion altogether:

“To really play Devil's advocate, I think there should be THREE tests!

1. The model initially has no name
2. The model cannot execute when it has no name
3. The model can execute when the name has been set.

You could squeeze that into two:

1. The model initially cannot execute
2. The model can execute when the name has been set

But I think the state transitions and constraints are clearer with three tests.”


Even with there now being the extra code for three functions, rather than one, this is of course the answer. It even satisfied the idea of one assert per method. The most difficult idea for me was not testing preconditions specific to a test if they were already tested in a another test .I have consequently modified my way of thinking and a lot of my test code.

Originally published in CVu July 2011.

Comments

  1. I think the issue here is that one (or more) of the "test" assertions are also "pre-condition" assertions.

    Things get messier as you start to stack these up. You start with state A, move into state B, then from there into state C etc. If your test case is to test state C but the only way to get there is via states A and B then you're going to want to test that each transition resulted in a valid state (I mean state and transition in a very general sense here - not necessarily anything to do with state machines).
    Start adding states D and E and it gets out of control (but perhaps you could argue that is a design issue).

    It appears that the suggestion above is to have individual tests for each state transition, but where you need to test something in a derived state you go through the earlier transitions without testing (on the basis that the individual tests will pick any failures up).

    This is ok, and probably as much as you can do with any C# test tools that I know of (other than lots of redundant additional assertions).

    But it still doesn't sit quite right with me either.

    So I did something about it :-) In the C++ world anyway. Actually it was based on one of Kevlin's ideas - but in Catch (my C++ unit test framework) you can have arbitrary nested sections. Each section can be considered a test case in its own right - but you can consider parent test cases as fixtures.

    This allows you to test your individual state changes in their own sections as you cascade down - then test your derived state at any level - all both orthogonally and in combination - with no redundant code.

    So you're not the only one, Paul.

    ReplyDelete
  2. I'd agree that having to worry about moving through lots of states indicates your class is possibly doing too much ... maybe there's a good model example in the code you've been testing?

    ReplyDelete

Post a Comment

Popular posts from this blog

Write Your Own Load Balancer: A worked Example

I was out walking with a techie friend of mine I’d not seen for a while and he asked me if I’d written anything recently. I hadn’t, other than an article on data sharing a few months before and I realised I was missing it. Well, not the writing itself, but the end result. In the last few weeks, another friend of mine, John Cricket , has been setting weekly code challenges via linkedin and his new website, https://codingchallenges.fyi/ . They were all quite interesting, but one in particular on writing load balancers appealed, so I thought I’d kill two birds with one stone and write up a worked example. You’ll find my worked example below. The challenge itself is italics and voice is that of John Crickets. The Coding Challenge https://codingchallenges.fyi/challenges/challenge-load-balancer/ Write Your Own Load Balancer This challenge is to build your own application layer load balancer. A load balancer sits in front of a group of servers and routes client requests across all of the serv

Bloodstock 2009

This year was one of the best Bloodstock s ever, which surprised me as the line up didn't look too strong. I haven't come away with a list of bands I want to buy all the albums of, but I did enjoy a lot of the performances. Insomnium[6] sound a lot like Swallow the Sun and Paradise Lost. They put on a very good show. I find a lot of old thrash bands quite boring, but Sodom[5] were quite good. They could have done with a second guitarist and the bass broke in the first song and it seemed to take ages to get it fixed. Saxon[8] gave us some some classic traditional heavy metal. Solid, as expected. The best bit was, following the guitarist standing on a monitor, Biff Bifford ripped off the sign saying "DO NOT STAND" and showed it to the audience. Once their sound was sorted, Arch Enemy[10] stole the show. They turned out not only to be the best band of the day, but of the festival, but then that's what you'd expect from Arch Enemy. Carcass[4] were very disappoin

Catalina-Ant for Tomcat 7

I recently upgraded from Tomcat 6 to Tomcat 7 and all of my Ant deployment scripts stopped working. I eventually worked out why and made the necessary changes, but there doesn’t seem to be a complete description of how to use Catalina-Ant for Tomcat 7 on the web so I thought I'd write one. To start with, make sure Tomcat manager is configured for use by Catalina-Ant. Make sure that manager-script is included in the roles for one of the users in TOMCAT_HOME/conf/tomcat-users.xml . For example: <tomcat-users> <user name="admin" password="s3cr£t" roles="manager-gui, manager-script "/> </tomcat-users> Catalina-Ant for Tomcat 6 was encapsulated within a single JAR file. Catalina-Ant for Tomcat 7 requires four JAR files. One from TOMCAT_HOME/bin : tomcat-juli.jar and three from TOMCAT_HOME/lib: catalina-ant.jar tomcat-coyote.jar tomcat-util.jar There are at least three ways of making the JARs available to Ant: Copy the JARs into th