Advancing
HW/SW Co-Verification Methodology
Table of Contents
Overview
The term hardware/software (HW/SW) co-verification was coined in
the mid 1990s to describe a process that allows embedded system
software to be executed on a simulated representation of the hardware
design. The simulated representation of the hardware has usually
meant a Verilog or VHDL description running in a logic simulator.
The concept of logic simulation has since been extended to allow
for other representations of the hardware that are not the final
implementation, including logic emulation systems and other prototyping
methods.
As designs become more complex, initial co-verification
solutions are running out of stream. At the same time, many complex
SoC designs are still using nothing more than a full-functional
simulation model of a microprocessor and waveforms to debug complex
hardware and software. A full-functional model of a microprocessor
fetching and executing code in a logic simulator is not co-verification
if the only means of debugging software are waveforms and assembly
language traces.
The difficulty of the problem can be seen by
looking at the vast array of alternatives available to verify both
hardware and software before first silicon is available. This paper
will help engineers break down the SoC verification problem into
digestible portions and help explain the best methods needed to
successfully bring an SoC design to market. The outline is derived
from the logical steps of the SoC software development process.
It maps the software development process onto the necessary hardware
execution and debugging technologies required for success. Once
the software development process is understood, a solution is proposed
that can be used as an outline for creation of an SoC verification
plan.
Benefits of Performing
Hardware/Software Co-Verification
The ultimate goal of co-verification is to get the entire system
-- hardware and software -- working prior to prototype by providing
better visibility into the behavior of both hardware and software.
Co-verification achieves this by providing two primary benefits:
- Software engineers have much earlier access
to the hardware design. This allows software designers to develop
code and test it concurrently with hardware design and verification.
Performing these activities in parallel shaves time from the project
schedule compared to the serial method of waiting for the prototype
to begin software testing. Moreover, the early involvement of
the software team results in a much better understanding of the
underlying hardware operation.
- Co-verification provides additional stimulus
for the hardware design. In fact, it can provide the true stimulus
that will occur in the embedded system. This improves hardware
verification when compared to using a contrived testbench that
may or may not represent real system conditions. Increased confidence
in the hardware design is invaluable.
By running hardware/software co-verification,
a wide range of problems can be found and fixed prior to silicon,
such as register map discrepancies, problems in the boot code, errors
in DMA controller programming, RTOS boot and configuration errors,
bus pipelining problems, and cache coherency mishaps. Some of the
errors will be software problems and some hardware related. Attacking
this laundry list of example problems must be done using a logical
and well-defined verification strategy. The key to a successful
project is to apply the correct tools to the problem and always
to understand what types of problems you are looking for and what
parts of the system are assumed to be working at each stage of the
project.
Overview of SoC Development
The SoC development process involves many facets. The traditional
notion of the hardware design being the device under test (DUT)
and the testbench being used to stimulate the DUT is outdated. The
DUT now includes software, and the testbench must verify how hardware
and software operate together. This paper is built around a common
approach used for SoC software development and maps the proper hardware
design technologies onto this five-step software process.
Five distinct types of embedded system software
have been identified, with software design proceeding in this order.
The software content (i.e., lines of code) increases with each step:
- System initialization software and hardware
abstraction layer (HAL)
- Hardware diagnostic test suite
- Real-time operating system (RTOS)
- RTOS device drivers
- Application software
The abstraction of the hardware and the assumptions
made about the stability and functionality also increases with each
stage of software development.
Four distinct methods used for the execution of the hardware design
description have been identified and are commonly used in SoC design:
- Logic Simulation
- Simulation Acceleration
- Hardware Emulation
- Hardware Prototyping
Each hardware execution method has specific
debugging techniques for both hardware and software associated with
it, each with its own set of benefits and limitations. These range
from the slowest execution method, with the most thorough debugging,
to the fastest, with less debugging. As an example, a common SoC
built around the popular ARM family of microprocessors will be described.
The Axis ReConfigurable Computing (RCC) platform used in the Xcite®
and Xtreme® products will be used to illustrate how to apply the
concepts discussed. This single platform is used for logic simulation,
simulation acceleration, hardware emulation, and co-verification.
The Axis XpertT HW/SW Co-Verification product will be used to demonstrate
software-debugging concepts.
The Debugging Loop
Beyond just tools and debugging methods
there is one more crucial area to cover, the human interaction between
hardware and software engineers. The optimization of this interaction
is equally as important as the optimization of the hardware and
software itself. This is best illustrated by describing a common
interaction model used in the early stages of many projects today.
A typical project will first partition the
system into hardware and software and follow with a register map
specification, usually written with a word processor, that describes
the software interface to the custom hardware. As time passes, these
registers are changed, but the specs are not updated. The written
descriptions are either incorrect or not clearly understood by the
software engineers. Hardware and software engineers have different
concepts and methods for describing the system functionality. For
example, software engineers don't usually have a concept of logic
signals being "active low."
To debug these systems, the process starts
when software engineers compile code and prepare memory image files
for execution. Because they are unfamiliar with logic simulation
and emulation tools, they pass the image file over to the hardware
or verification engineers who run the test.
When the test fails, the verification engineer
tries to isolate the problems, and presents the result to a design
engineer who is familiar with the specific part of the design that
may have a problem. The design engineer analyzes and fixes the problem
and runs the test again.
If the problem is in the software, then the
software engineer must be called in to inspect the problem. For
the software engineer to debug the problem, he needs the help of
the verification engineer who understands waveforms from the simulation
and can correlate it to the software.
Finally, the software engineer generates a
fix, and the process starts all over. This iterative cycle of throwing
results over the wall among three groups struggling to work together
slows down the progress of the project.
Figure 1: Typical Debugging
Cycle
Co-verification aims to tighten this debugging
loop by allowing all groups to work independently and concurrently
as much as possible. Any tools or methods that empower software
engineers to find and fix problems with less intervention by verification
and hardware design engineers are of great benefit to the entire
project.
Given this background, the next section will
move into the embedded software phases of SoC design and outline
best practices for co-verification success.
System initialization and
HAL
The first coding task is to write and test the processor initialization
software. This code includes configuring the operating modes and
peripherals (things like cache configuration, memory protection
unit or MMU programming, interrupt controller configuration, timer
setup, and DRAM initialization). This is a tedious coding task using
assembly and C languages, but important since all other software
relies on it, including the final product.
The hardware abstraction layer is the next
layer of software that works with the initialization code to provide
a common interface for higher-level software to use for hardware-specific
functionality after the system is initialized. The HAL abstracts
the underlying hardware of a processor architecture and the platform
to a level sufficient for the RTOS kernel to be ported onto the
platform. For example, consider a system that has an interrupt controller
with a prioritized interrupt scheme. The RTOS porting engineer would
like to change the priority using a simple C function and a priority
level, like ipl(5), to set the priority level to 5. The detail of
disabling interrupts and using an atomic read-modify-write sequence
to change the interrupt controller registers and then enabling interrupts
again are not interesting or necessary.
Many complex system-on-a-chip (SoC) projects
use nothing more than a full-functional model of the microprocessor
core in a logic simulator to write and debug this code. Software
debugging with waveforms requires a true guru who understands hardware
and software and can disassemble instructions in his head using
instruction fetches on the data bus.
For the ARM SoC example, the ideal debugging
solution for early development of system initialization and HAL
code is one based on a cycle-accurate instruction set simulation
model tightly coupled to a logic simulator containing the SoC hardware
design. This provides interactive, graphical software debugging
for the software engineer to single step through the code and verify
register and memory contents with excellent control. Simulation
performance is less important because the code must be verified
line-by-line and the number of lines of code is relatively small.
A standard Verilog simulator such as Xsim® is more than adequate
to execute the hardware design and provides good visibility into
the hardware. Interactive software debugging provided by Xpert's
graphical debugger interface is the ideal way to control software
execution and very software operation.
It is common for many hardware bugs to be encountered
during this phase since it generally occurs soon after the blocks
of the SoC have been integrated into a full-chip simulation and
is the first time all of the subsystems are working together using
the defined hierarchy of shared busses.
Diagnostic Suite
Once the initialization software and HAL are stable, the next phase
of software development consists of developing a detailed test suite
for the hardware design. In the past this usually took the form
of a hardware testbench. While testbenches are still necessary to
provide stimulus for external interfaces such as networking protocols,
the software now serves as the testbench for the CPU core bus. A
comprehensive set of diagnostic tests should be developed to verify
each subsystem and peripheral. This starts with the memory subsystem,
progresses to interrupt testing, and then moves to other IP blocks
like timers, DMA controllers, video controllers, MPEG decoder, and
other specialty hardware. Most of these tests do not see their way
into the final product but they are very important because they
build the case for a solid hardware design. Creating the programs
gives the software engineers a very good understanding of the hardware
and serves as a chance to learn about the hardware specifics in
a more secluded environment.
It is during this development of diagnostic
tests that the logic simulator begins to become the bottleneck of
the verification environment. As tests run longer and longer and
the number of tests increases, it becomes increasingly difficult
to both verify the entire hardware design and to continue to run
the old tests as hardware and software errors are fixed. This phase
is also the most crucial since it is where most of the hardware
bugs are found. Debugging tools for both software and hardware at
this stage are very important.
The ideal solution uses simulation acceleration
to increase the performance of logic simulation over what is a possible
using an ordinary software simulator. A simulation environment running
at 10 to 100 Hz is not fast enough for software engineers to run
and tests anymore. Moreover, the memory optimization techniques
commonly used for co-verification are not useful because the main
purpose of the diagnostics is hardware verification. To abstract
the hardware using time domain techniques and stray from a cycle-accurate
simulation goes against the purpose of the tests. A simulation acceleration
system that runs at speeds of 1 to 10 kHz is the ideal platform
for simulation performance and debugging.
During this phase of the project, debugging
is equally as important as speed. Traditional hardware debugging
techniques such as waveform dumping are far too inefficient. The
VCD-on-Demand feature of RCC allows engineers to extract all node
history values from any point in simulation without re-simulating
from time zero. This is another reason to deploy simulation acceleration
during this phase. The combination of Xcite for simulation acceleration
and Xpert for software debugging is the best way to create and verify
the hardware diagnostic suite.
As the test suite grows many tests are run
in parallel, probably overnight in a batch environment. If specific
tests fail, engineers must analyze the results. While hardware engineers
are familiar with post-processing debugging methods, software engineers
are not. A debugging method that allows software engineers to inspect
the test results without re-running the test is ideal. For the ARM
example a technique called Instant ReplayT is provided by Xpert
for ARM processors.
Instant Replay Software
Execution using Instruction Set Simulators
During batch simulation, a compressed file is created containing
bus transactions at the processor interface. The memory transactions,
including address, data, and simulation timestamp along with interrupt
information, are "recorded" into this file. After the
simulation is complete, software engineers can start the instruction
set simulator and software debugger and "re-run" the software
execution sequence. However, this time instead of interacting with
hardware simulation, the results are read from the recorded file.
This "playback" of the bus interface replicates the exact sequence
of software execution.
Figure 2: Using Instant
Replay
Because the simulation now runs at MHz speeds,
software engineers can re-run the software as many times as needed
to find the problem -- without looking at any simulation waveforms.
The simulation timestamp is also provided at any time to help correlate
the software and hardware execution. This record and playback methodology
makes Xpert the ideal way to debug long simulation tests that make
interactive debugging unproductive.
Real-time operating system
(RTOS)
One of the great benefits of this methodology and the diagnostic
test suite is that the hardware, initialization sequence, and HAL
are assumed good (pretty close, anyway) when the real time operation
system work begins. The first assumption of the RTOS engineer is
that the hardware is stable.
The initial RTOS work consists of just getting it to boot on the
platform. How big a task the RTOS is depends on how standard the
platform is and how well the HAL was designed. During the initial
porting phase, device drivers for application specific hardware
may be missing, but the RTOS can still boot.
The RTOS port is an ideal place to take advantage of memory optimizations
commonly used in co-verification such as software memory models.
Software memory models use a memory that is tightly coupled to the
instruction set simulator to retrieve instructions at a much faster
rate than using logic simulation. The result is less simulation
detail on how the ARM SoC would work but increased performance.
Since the instruction fetch path is well verified, this is where
using the memory optimizations make sense, not back in the diagnostic
test suite as a workaround for a low logic simulator. Xpert includes
important memory optimizations for software memory models tightly
coupled to the instruction set simulator to increase software execution
performance.
Device Drivers
The final stage of verifying the hardware design and system software
is to develop and test the remainder of the device drivers for the
RTOS. For the RTOS porting work and the device drivers, the first
attempt should be done using simulation acceleration mode. Once
the drivers are verified a faster hardware execution method can
be used.
Once the RTOS is booted and stable, future work can be done using
a faster execution method such as an emulation system or a prototype.
Here debugging is a bit more difficult, but the number of hardware
bugs is very small. The choice of emulation versus prototyping is
a complex one. Several factors are involved but the primary factors
revolve around the number and type of required interfaces to exercise
the SoC and the benefits gained by running at prototyping speeds.
A tradeoff decision is required to assess the benefits of prototyping
speed such as 20 MHz versus emulation speeds of less than 1 MHz
in exchange for the debugging difficulty. The decision also depends
on the design confidence in the testbenches used to verify interfaces
in simulation acceleration mode. The Xtreme product from Axis provides
interfaces to target systems used to verify real-world interfaces
and increase design confidence. Xtreme provides the necessary performance
to execute the RTOS and device drivers. The combination of Xtreme
with a target board containing the ARM silicon is the ideal solution
to obtain the next level of performance, 1-2 orders of magnitude
over that of acceleration mode.
Application Software
Once the platform is complete, it is time to test application software.
At the application level the hardware is assumed correct and becomes
more like workstation programming. If the application crashes it
can be safely assumed it is a software problem, not something wrong
with the hardware. Projects that are too large for software-based
logic simulation have long turned to logic emulation systems such
as Xtreme to gain speeds of up to 1 MHz and access to hardware interfaces
to real world stimuli. More recently, a need has developed to run
at speeds of 20 MHz or more. This has fueled the use of prototyping
for higher speeds at the expense of visibility. These platforms
are best for application development. Sometimes even waiting for
silicon is best for applications: this depends on confidence in
the hardware design and whether everything was verified.
Applications usually want to interface to real network traffic,
see things on the screen, and use the pointer or mouse. During application
development, hardware and lower-level software bugs are few and
far between and the software engineer is focused on providing robust
applications with differentiating features for end users.
Summary of Verification
Flow
Below is a summary of the ARM SoC verification flow described using
Axis products.
- Assemble full chip RTL netlist and testbench
for Xsim simulation. ARM SoC simulation should be constructed
to allow both the Xpert ARM model and the ARM design signoff model
(DSM) to be used.
- Run Xsim to verify operation of device sub-systems
including memories and peripherals using the software debugger.
Without writing or loading any software targeted reads and writes
can be generated from the debugger to make sure all memories and
registers are working without any bus conflicts.
- Test and debug ARM initialization software
and HAL using Xpert and Xsim. Performance in software simulation
is typically less than 200 cycles/sec, but code is small and graphical
debugging provides control and visibility of the design.
- Develop and test a complete suite of diagnostic
tests to obtain maximum hardware design coverage. This is where
most of the hardware design is verified and debugged. As test
suite grows, transition the design to RCC for simulation acceleration.
Now performance is improved to 1,000 - 5,000 cyles/sec with full
cycle accuracy. Utilize Instant Replay to help debug test failures.
When all test pass, the hardware design should have very few bugs
remaining and can be assumed "known good".
- Port real-time operating system (RTOS) and
associated device drivers. This work begins in acceleration mode
with Xpert. Using Xpert's memory optimizations performance is
approximately 20,000 - 50,000 cycles/sec. Once the RTOS is booted,
transition to emulation with Xtreme and an ARM target board for
higher performance and testing of other interfaces such as video
display or communication protocols. Now performance is approximately
200,000 cycles/sec.
- Develop application software using Xtreme
or move to prototype depending on performance requirement and
number of application developers requiring access to hardware.
The key to this verification flow is the smooth
transition from simulation to acceleration to emulation. Axis is
in a unique position to provide this complete flow for SoC verification
using RCC technology as the hardware execution platform, VCD-on-demand
for hardware debugging, and Xpert for software execution and debugging.
This allows the verification tasks to be done using logical transitions
between project phases. One example of a common mistake is to attempt
to eliminate the simulation acceleration mode of execution by using
an emulation system that is disjoint from software simulation. When
this occurs software simulation runs out of steam during the diagnostic
test suite phase and the design is forced to move to emulation too
early. Now time is wasted trying to debug many hardware problems
as well as software bugs in a difficult emulation environment.
Conclusion
Project teams must develop a common sense plan to incrementally
test and verify hardware and software together, right from the start
of the project. Too often, project teams use a seemingly random
approach to developing different types of software and try to run
it using many different execution modes in parallel, just hoping
one of them will work and prove that everything is ok. There is
a great struggle today to decide how and when to deploy specific
products for co-verification, simulation acceleration, emulation,
and prototyping. The risk is too great to try to haphazardly mix
and match tools late in the project. Engineers have nagging questions
about whether emulation will really work and how fast it will go.
They also wonder if the design will really fit into large FPGAs
for a prototype, and if the prototype does run, whether it will
be possible to debug it.
The ideal solution provides a single integrated
system for logic simulation, simulation acceleration, and system
emulation along with the concurrent debugging capabilities of both
hardware and software. Advancements in co-verification will come
by providing a smooth transition across these platforms with the
right debugging tools for the job.
|
