RSS
RickyHo

Test as a Spec

Wed, Nov 4, 2009

RickyHo

One of a frequently encountered question in enterprise software development is where does the hand off happen from the architect (who design the software) to the developer (who implements the software). Usually, the architect designs the software at a higher level of abstraction and then communicate her design to the developers, who take the idea and transform it into implementation-level code.

The hand off happens typically via an architecture spec written by the architect, composed of a set of UML diagrams. The programmers read these diagram and create code from there.  Although most CASE tools provide code generation capability, it is not commonly used because the level of abstract in the design and code implementation are quite different. After the developer digest the meaning of the UML diagram, he/she go ahead to write code.

What is the problem ?

However, the progression is not as smooth as we expect. There is a gap between the architecture diagrams and the code which requires a transformation. During this transformation step, there is a possibility of mis-interpretation, wrong assumption. Quite often, the system ends up being wrongly implemented due to mis-communication and misinterpretation which can happen because the architect hasn’t described his design clear enough in the document, or because the developer is not experienced enough to fill in some left-out detail that the architect consider obvious.

One way to mitigate this problem is to have more design review meetings, or code review session to make sure what is implemented is correctly reflecting the design intention. Unfortunately, I found such review sessions are usually not happening either because the architect is too busy in other tasks, or he/she is too “hands-off” and doesn’t want to work at the code level.  It ends up the implementation doesn’t match the design. Quite often, this discrepancy is discovered at a very late stage and left no time to fix. While developers continuously patching the current implementation for bug fixing or adding new features, the architect start to lose control on the architecture evolution.

Is there a way for the architect to enforce her design at the early stage given the following common constraints ?

  1. The architect cannot afford frequent progress/checkpoint review meetings
  2. While making sure the implementation compliant with the design at a higher level, the architect doesn’t want to dictate the low level implementation details

Test As A Specification

The solution is to have the architect writing the Unit Tests (e.g. JUnit Test Classes in Java), which acts as the “Spec” of her design.

Note the “component” is an important artifact. The architect will provide a responsibility description of each component.  Each component expose 2 sets of interfaces

  1. Facades - defines the interface that this component “provides”
  2. Colloborators - defines the interfaces that this component “uses”

But more importantly, the architect also write unit tests against these interfaces.

By writing concrete Unit Tests against the “Facades”, the architect fully specify the external behavior of the system from the perspective of the component’s client.

By expressing “Collaborator” in terms of Mock Obects, the architect fully specify the expectation of its underlying supporting objects.

In other words, the unit test defines how this component interact with external parties, including its client (how this system will be used), as well as the collaborators (how this component uses other components).  It becomes a very concrete design spec of the component and leave no room for mis-interpretation.  The unit tests serves a number of important purposes

  1. It is the conformance test of the implementation (even in future evolution)
  2. It is a usage example document
  3. It is a design specification (now developers knows exactly when the architect is expecting)

Note that the architect is not writing ALL the test cases. Architecture-level Unit Test are just a small subset of the overall test cases specifically focus in the architecture level abstraction. This unit test are specifically written to ignore the implementation detail so that it gives enough freedom for the developers to choose how the implementation should be done. On the other hand, the architecture itself becomes more stable as it will not be affected by future changes of implementation logic.

Other than the architectural unit tests coded by the architect, the developers provide a different set of implementation level unit tests.  Usually, this set of “Impl Level TestCase” which change when the developers change the internal implementations, but this won’t affect the “architectural test cases”. By separating these 2 sets of test cases under different categories, both sets of test cases can evolve independently when different aspects of the system changes along its life cycle, and resulting in a more maintainable system as it evolves.

To illustrate, lets go through an example using a User Authentication system. There maybe 40 classes that implements this whole UserAuthSubsystem. But the architecture-level test cases only focused in the Facade classes and specify only the “external behavior” of what this subsystem should provide. It doesn’t touch any of the underlying implementation classes because those are the implementor’s choices which the architect doesn’t want to constrain.

User Authentication Subsystem Spec

Responsibility:

  1. Register User — register a new user
  2. Remove User — delete a registered user
  3. Process User Login — authenticate a user login and activate a user session
  4. Process User Logout — inactivate an existing user session

Collaborators

  1. Credit Card Verifier — Tell if the user name match the the card holder
  2. User Database - Store user’s login name, password and personal information
public class UserAuthSystemTest {
  UserDB mockedUserDB;
  CreditCardVerifier mockedCreditCardVerifier;
  UserAuthSystem uas;

  @Before
  public void setUp() {
    // Setup the Mock collaborators
    mockedUserDB = createMock(UserDB.class);
    mockedCardVerifier =
        createMock(CreditCardVerifier.class);
    uas =
        new UserAuthSubsystem(mockedUserDB,
                              mockedCardVerifier);
  }

  @Test
  public void testUserLogin_withIncorrectPassword() {
    String uName = "ricky";
    String pwd = "test1234";

    // Define the interactions with Collaborators
    expect(mockUserDB.checkPassword(uName, pwd)))
                     .andReturn("false");
    replay();

    // Check the external behavior is correct
    assertFalse(uas.login(userName, password));
    assertNull(uas.getLoginSession(userName));

    // Check the collaboration with collaborators
    verify();
  }

  @Test
  public void testRegistration_withGoodCreditCard() {
    String userName = "Ricky TAM";
    String password = "testp";
    String creditCard = "123456781234";
    expect(mockCardVerifier.checkCard(userName,creditCard)))
                           .andReturn("true");
    expect(mockUserDB.addUser(userName, password)));
    replay();
    uas.registerUser(userName, creditCard, password));
    verify();
  }

  @Test
  public void testUserLogin_withCorrectPassword() { .... }

  @Test
  public void testRegistration_withBadCreditCard() { .... }

  @Test
  public void testUserLogout() { .... }

  @Test
  public void testUnregisterUser() { .... }
}

Summary

This approach (”Test” as a “Spec”) has a number of advantages …

  1. There is no ambiguation about the system’s external behavior and hence no room for mis-communication since the intended behavior of the system is communicated clearly in code. Such code sitting at the boundary also tends to be more readable.
  2. The architect can write the TestCase at the level of abstractions she choose. She has full control in what she wants to constraint and what she wants to give freedom.
  3. By elevating architect-level test cases as the spec of the system’s external behavior. They become more stable and independent of implementation level. This makes the architecture more stable.
  4. This approach force the architect to think repeatedly from an external perspective, what is the “interface” of the subsystem and also what are the collaborators. So the system design is forced to have a clean boundary.
  5. The test cases can also serve as the usage examples

Popularity: 36% [?]

, , ,

Leave a Reply