Functional Verification Automation for IP
Bridging the Gap Between IP Developers and IP Integrators


Introduction

The development and integration of intellectual property (IP) in the form of large functional blocks—or cores—is an essential part of an increasing number of today's IC design strategies. Numerous benefits justify this approach. Properly managed, the use of pre-designed, pre-verified IP blocks can cut man-months off of the design cycle, provide opportunities for design reuse in future systems, and where applicable, ensure compliance with complex standards and protocols.

Unfortunately, the emergence of the IP-based design methodology has done as much to exacerbate design and verification problems associated with systems-on-chip (SOC) designs as it has to alleviate them. For example, how does the IP developer pass on knowledge of an IP block with sufficient breadth and depth to ensure that the integration process will progress smoothly and quickly toward a successful implementation? On the other hand, what assurance does the IP integrator have that a recently purchased core has been adequately verified? Further, once the IP is integrated within the system, and a bug emerges during simulation, how does one determine if it originated in the IP or in the surrounding circuitry?

Functional verification automation addresses these and other verification problems by helping to bridge the gap between IP developers and IP integrators, whether they work in separate departments or in separate companies.


The IP ERA: Building Content and Complexity

The prevalent trend toward the use of IP in SOC designs was born primarily out of two compelling needs: to reduce the time and cost required to fill up the enormous gate capacities inherent to deep-submicron processes, and to amortize the cost of creating functional blocks whose content offers value as reusable design elements. Unfortunately, the advent of massive IP-based ICs has also given rise to extraordinarily complex design and verification problems.

The electronic design automation (EDA) industry has made great strides in alleviating some of these design problems through a variety of high-performance tools and methodologies that improve and extend design automation, raising the design effort to higher levels of abstraction. However, advances in verification have centered primarily on increasing the speed of simulation, not on automating the verification methodology as a whole. Thus, the difficulties one encounters when verifying these large, complex, IP-based designs remain relatively unaddressed.

Specman™, Verisity's functional verification automation product, directly addresses the problems that arise when verifying correct integration of IP. To understand how Specman accomplishes this, one must understand the nature of the verification bottleneck. As designs grow more complex, the verification problems they pose grow exponentially—as designs double in size, the verification code can easily quadruple. As a result, the typical verification effort consumes as much as 50 to 70 percent of the entire engineering budget.

Unfortunately, neither the schedule nor the available engineering resources offer much in the way of flexibility. More importantly, the cost of missing a time-to-market schedule can force design teams to prematurely terminate the verification effort. This leads to incomplete and inadequate functional verification, creating the potential for a pattern of debug cycles, re-designs and re-spins that erode the profitability and quality of the end product. Thus, IC design quality becomes a function of the verification schedule, rather than verification metrics.

Functional verification automation is an emerging methodology for functional verification that solves many of the problems associated with verifying large, complex designs. An effective application of this methodology provides three essential capabilities to help break through the verification bottleneck:

  • it automates the verification process, reducing by as much as four times the amount of manual work needed to develop the verification environment and tests;
  • it provides functional coverage analysis capabilities to help measure the progress and completeness of the verification effort;
  • it raises the level of abstraction used to describe the environment and tests from the RTL level to the architecture/functional specification level.

As bad as verification is for today's complex designs, it is even worse for IP-based designs. Besides dealing with the challenge of size and complexity, an effective verification methodology for IP-based designs must also solve the verification problems of both the IP developer and the IP integrator.

IP developers and IP integrators have two very different views of the world, and therefore, two separate sets of verification problems. The IP developer may be intimately familiar with the micro-architecture of the block of functionality they have created, but have only a fuzzy idea about what the target system will look like. Conversely, the IP integrator knows the target system in detail, but the IP itself is a "black box" often buried deep within the design, offering the integrator little, if any, controllability or visibility to the primary inputs and outputs of the IP.

IP Developer

IP Integrator

The IP developer has a detailed understanding of the microarchitecture, but an incomplete view of the target system.

The IP integrator understands the system, but sees the IP as a black box, often with little or no visibility or controllability at the IP ports.


The IP Verification Problem

The IP developer needs to create high-quality IP to overcome their customers' apprehensions about placing something that was designed by somebody else into the critical path of their success. To the developer, successful verification means being able to assure the customer (the integrator) that there are no quality problems at the IP block level, and that there will be none once the block has been integrated with the rest of the system. The ability to offer substantiated assurance is critical to the IP developer, because their success hinges on the customer's design making it to volume production. This also means they need to help solve the IP integrator's verification problems.

The IP integrator has a multifaceted verification problem. At the outset, they are looking for verifiable high-quality IP. Otherwise, any sign of problems in the design, and the IP becomes the immediate suspect. This can create a major support burden for the IP developer, and a very slow loop for debugging the design that erodes the profits for both sides.

The integrator also needs to verify at every stage of the design that the integration process complies with the rules set forth by the developer. These rules attempt to capture information about the capabilities of the IP's architecture and the assumptions made about the target system's interaction with the IP. Often, these rules accompany the design in the form of a manual of guidelines, which may or may not cover the exact usage scenario being used by a particular target system. The integrator must become as familiar as possible with the integration guidelines, despite having little or no access to the internal workings of the IP. Thus, the integrator and the developer must establish a common basis of communication if they are both to succeed.

Having completed the integration process, the integrator faces the difficult task of creating system-level (or chip-level) tests that sufficiently exercise both the IP and the surrounding system. Problems arise from a number of areas:

  • the IP may not sit at the boundary of the system, making access a problem;
  • the integrator may have little knowledge, much less control, of the user modes or scenarios generated by the system-level tests, making the IP's response highly unpredictable;
  • verification coverage of the IP is virtually indeterminate.

What Worries the IP Developer

  • Need for high-quality verification of their IP with limited knowledge of the target system.
  • Need to ensure customer integration success.

What Worries the IP Integrator

  • Need for high-quality IP
  • Need to ensure customer integration success.
  • Need to make sure the chip-level tests exercise the IP "enough"

The Specman Elite Solution for IP Verification

The IP Developer
The functional verification automation methodology enabled by Specman Elite offers verification assistance both for the developer and the integrator of IP. At the development level, Specman Elite addresses the developer's need to create high-quality IP in four ways:
  • by increasing productivity;
  • by automating the generation of tests, including those that address corner cases;
  • by providing effective randomization capabilities to enable the engineer to test for the unexpected;
  • by enabling functional coverage analysis which measures the progress of the verification effort against a functional test plan and directs the verification process to ensure that every test is adding new and important coverage.

The one feature Specman Elite offers that most significantly contributes to increased productivity is automated test generation. Armed with a constraint-driven test generator, a functional verification automation methodology can slash weeks, or even months off of the verification schedule for an IP block.

Specman Elite's constraint-driven generator gives the IP developer full control over the generation process. It can generate tests that are completely random, completely deterministic or anywhere in between. Instead of randomizing only what the developer specifically instructs the system to randomize, the constraint-driven generator take the "infinity-minus"approach. It randomizes everything except what the engineer specifically constrains the generator not to randomize. This allows the generator to approach any given test scenario from multiple paths. Since IP developers have limited knowledge of the target system, the randomization in this methodology is particularly important to allow them to test for unanticipated user scenarios.

Specman Elite also greatly eases the process of making tests self-checking. Particularly since the IP may be deeply embedded inside the target system, it is important to supply checkers that check both data integrity and temporal behavior. Specman Elite contains a rich set of temporal constructs that make it easy to create checkers to capture complex protocols.

Functional coverage analysis gives the IP developer a clear indication of which functionality within the IP block has been exercised, and more importantly, which functionality has not. This eliminates superfluous or redundant tests, ensuring that every test adds coverage.
The functional coverage reports generated by Specman Elite help monitor the progress of the IP verification process, providing a clear indication of the state of the verification. Ultimately, the coverage reports enable the developer to determine when the IP is ready to be delivered to the integrator. Functional coverage reports also enable the IP developer to verify, record and communicate the effectiveness of the tests to customers (integrators).

The impact on the IP verification schedule when using this approach is significant. It eliminates much of the manual-intensive work of developing deterministic tests, saving many man-months of frustrating work, and ultimately, eliminating the risk of sending the IP to the integrator with part of its functionality untested.


Verisity's Specman Elite automates functional verification.

The IP developer also has a vested interest in helping its customer's succeed. Specman Elite helps solve the IP integrator's problems by making the IP integration rules executable and by enabling functional coverage analysis. Specman's functional coverage analysis feature allows the IP developer to define exactly what functionality of the IP should be monitored and reported during system or chip-level tests. To accomplish this, the IP developer supplies a toolkit, which captures both the rules, as checkers, and the interesting coverage scenarios in executable form and delivers this to the IP integrator.

The IP Integrator
Typically, the IP developer runs extensive regression tests, often taking weeks to complete, before releasing a block of IP. Upon delivery to the customer, the developer often provides a greatly condensed subset of the regression test to provide some assurances to the customer that the IP operates within the context of the developer's simulation environment. Since the customer has no way to measure the effectiveness or completeness of the verification effort, the developer may also provide a copy of their functional test plan.

Specman Elite enables the IP developer to give the customer an executable test plan with coverage reports from the run-time execution of the simulation. The reports show exactly what was done in the verification—a very straightforward capability inherently provided by Specman Elite.

Because the target system can be so complex, rather than just providing an exhaustive book of rules governing the assertion and timing of signals, Specman Elite allows the IP developer to deliver executable checkers around the boundaries of the IP which capture data dependencies and temporal rules. Thus, Specman Elite can check any arbitrarily complex, mode-dependent, multi-cycle scenarios that the IP developer specifies. This process is completely invisible to the IP integrator.

Specman Elite essentially delivers two types of checks:

  • boundary checks to ensure that the IP has been integrated correctly;
  • internal checks which capture the developer's own internal checks to make sure that if there is a problem with the IP, the integrator can quickly resolve it.

Delivering both boundary checkers and internal checkers enable IP integrators to quickly pinpoint whether a problem in the simulation is due to a failure in the integration or a problem in the IP itself. These checks help eliminate any finger pointing between the developer and the integrator should a bug appear, and help to reduce the cost of support. Further, when there is a problem with the IP, these checks help direct the developer to the exact location of the problem.

Interface Spec:
Slave must assert ACK 1 to 5 cycles after REQ detected

Checker:
expect @req => { [1..5]; @ack}@clk else
    dut_error("Slave must assert ACK 1 to 5 cycles after REQ detected");

Interface Spec:
The DUT sets Parity Error Detected bit when target asserts parity_err on DUT data phase

Checker:
expect @parity_err_assert => @set_err_bit within
        interval(@parity_err_assert, @end_data_phase)@clk
    else dut_error("DUT must set Parity Error Detected bit when
                              target asserts parity_err during data phase");

Specman Elite allows easy creation of data and temporal checks.


Even with checkers sitting at the boundaries of the IP, if the integrator doesn't run the right test scenarios, they may not find an integration problem. Using Specman Elite, the IP developer can deliver the important coverage scenarios, which are monitored during system or chip-level testing. These give valuable information to the IP integrator in where there may be holes in the integrator's coverage of the IP. This helps the IP integrator determine whether all of the user modes and important multi-cycle scenarios were exercised. The IP developers may also choose to supply coverage scenarios of internal corner cases and state machines (both state and transition coverage), allowing even more visibility into the real coverage of the IP.

IP Developer

IP Integrator

Complete coverage before shipping.
Visibility into holes in system-level test.

Pure IP Program

Verisity works with IP developers to help develop toolkits they can deliver to their end customers. The intent of the toolkits are to provide checkers and coverage capability that are pertinent to the integrator's specific application. The toolkits are typically configurable, which is especially beneficial for IP developers offering soft IP that has been customized for a particular application, or hard IP where only portions of the capabilities are used. The tool kits enable integrators to access error messages from the checkers, as well as coverage information that will help the integrator identify problems in the integration and holes in the system or chip-level tests.

The tool kits contain complete documentation, executable checkers, coverage scenarios, and a version of Specman Elite - called Invisible Specman - that does not require the IP integrators to learn or purchase Specman Elite.

Verisity's Invisible Specman

  • Allow IP vendors to deliver a run-time version of Specman with subset capability.
  • Invisible Specman allows delivery of data and temporal checks, and functional coverage at boundaries as well as internal
  • Invisible Specman requires no learning curve, and no knowledge of Specman Elite necessary

Conclusion

For the IP developer, verifying IP quality depends largely upon the degree to which the simulation environment accurately represents actual system-level operating conditions—the area with which the developer is least familiar—and the completeness of functional coverage attained. Ultimate success depends upon the developer's ability to communicate the "rules" of integration to its customers and enable smooth integration into a working chip.

For the IP integrator, success depends upon strict adherence to the developer's "rule book" for proper integration of the IP—the area with which the developer is least familiar—and effective coverage of the IP during system or chip-level tests.

Both parties face time and resource constraints that place a premium on productivity in this, the most time-consuming effort in the product development cycle. Moreover, both the IP developer and the IP integrator fight a continual battle to keep support costs to a minimum, and share a vested interest in accelerating the time-to-volume. Functional verification automation addresses the concerns of both the IP developer and the IP integrator, greatly reducing the risk in IP-based designs.

 

Verisity Design, Inc.  •  2041 Landings Drive  •  Mountain View, CA  94043  •  phone: (650) 934-6800  •  Fax: (650) 934-6801  •  www.verisity.com/

Specman Elite,  SureCov, Verisity, and the Verisity logo are trademarks of Verisity Design, Inc. All other trademarks are the exclusive property of their respective holders. ©1999 Verisity Design, Inc.
© Copyright 2005 Verisity Design, Inc. All rights reserved. Privacy Policy.