Testers vs. Developers

Assertions can be derived as soon as a specification exists, however in this code-based methodology, assertion localization does not occur until after code testability is measured. Thus since testability measurements must be performed before the assertions are derived, this process cannot be performed until later in the software development life-cycle.

However, knowing that, for example, variable a needs to be tested after assignment statement 100, does not indicate what constraint the assertion should be testing for (to determine whether a problem with a has occurred). Hence our methodology has only addressed one part of the oracle/assertion problem, specifically placement, not derivation. Someday we would like to link this tool up with a formal specification language, and automatically generate the assertions, but that remains a research goal for the future.

As mentioned earlier, testers are likely to be incapable of deriving correct assertions, while developers are more likely to derive assertions that mimic the semantics of the code already there. If the developer does not understand the specification, their assertions will be incorrect. That is to say that if a developer's code is faulty, their assertions will almost certainly be faulty.

This brings us to our final recommendation for how to team developers and testers in a partnership to strategically derive and embed assertions. Let the testers find where assertions are needed, and leave it to the developers to determine what the assertions should be. Note that this is different than the case where the developer determines both where to place the assertions and what they should be. Here, the developer is being forced to derive assertions that the tester needs for places in the code where the developer might not be as sure as to what the assertion should be. If this happens, it forces developers to dig deeper into the code and requirements than they might have already done. This plays a similar role as code inspections, except that the person digging into the code is also the person that is likely to have written the code. Although this is not a foolproof solution, it is the best recommendation that we can provide at this time. Having developers spend more time comparing their previous understanding of the code to the existing requirements can only serve to improve the code's quality. Even if the assertion that the developer derives does not detect an error, the fact the the developer is forced to derive the assertion will increase the likelihood that the developers themselves find errors, because they are forced to get more familiar with less familiar internal computations in the code.