Verification Vertigo

In a previous blog, I talked about the differences between verification and validation. What can confuse those definitions even more is when we start looking at verification in a hierarchical manner. This hierarchical process also brings in another fundamental distinction, namely that of positive and negative verification. In the book ESL Design and Verification I defined those terms in this way.

You either demonstrate or prove that a design performs a necessary task—positive verification—or you demonstrate or prove that bugs do not exist—negative verification.

This would appear to be similar to the distinction between verification and validation, except that we have to remember the principle source of the second model against which we are comparing the design. Assume for a moment, that an executable specification for the system exists. When we verify the specification, we are doing validation. When we are comparing an implementation against the specification, we are doing verification. In order to do implementation, we first have to break the specification down into smaller parts, unless the system was small enough to start off with. In addition, partitioning and mapping may also have been performed. As we break the spec down, there is a chance that errors will creep in, especially since there is an almost total lack of tools or automation to help with this task. So by the time we get to the block level, several such decompositions steps may have taken place. There is thus a finite chance that errors will have crept into the block specification, or that elements of the specification have been omitted.

So now if we verify an implementation against that derived specification, we could prove that all bugs have been eliminated, but that does not mean that when put into the context of the system that it will work. In other words we need to either verify a block in the context of the system, or we have to validate that the sub-specification was correct. This is true for all levels of verification and integration back up to the full system, when the definitions for positive, negative, verification and validation fall back inline with each other.

Positive verification demonstrates that a design requirement is met by verifying it against the original specification. Negative verification exposes any flaws in the implementation of a requirement and thus compares it against the local specification. Most methodologies in use today focus on negative verification because of their bottom-up nature.

When we consider designs that include a large amount of design IP, be it from third party companies or design components that are reused internally, this situation is magnified. When IP is being verified, it is not known exactly how it will be used, so all behaviours are verified equally. But, some or many of those behaviours may not be used in the design, potentially wasting verification time. Consider for a moment a design that is to be based on a platform. That platform is likely to contain many components such as processors, memories, peripherals and the logic necessary to integrate them both at the hardware and software levels. The platform manufacturer has hopefully performed a lot of verification on this platform so let us assume that it does indeed contain no bugs. Does this mean that all designs placed on that platform will operate correctly? If the platform matches every need of the design and in a way that has been correctly understood by the user of that platform then the answer should be yes. But there likely were a lot of assumptions and loose statements that qualified its application, and the reality is that there will be mismatches in what the platform provides and what is required. In a verification flow, we would thus still have to show that each feature required in the final product is indeed properly implemented when the system has been assembled.

However, we cannot rely solely on positive verification either because it suffers from the problem that if a new behaviour is added or changed—perhaps due to a software fix—you have not verified that it will operate correctly in the hardware. Given the truth to the adage “if it hasn’t been verified, then it does not work,â€? this presents a problem. Hence, a balance is needed. Positive verification is mainly employed in the sections preceding implementation verification: post-partitioning analysis and verification. Implementation verification is the time to apply negative verification and to show that the positive verification has not been broken by the transformation. By thinking of verification from both the positive and negative perspectives it becomes possible to better schedule some of the verification tasks. Features that are most vital to a product can and should be verified before effort is expended on less important features. This ensures that if time pressures force the chip to be released before verification is finished, those features most likely to cause problems have been adequately verified.