The open standard ISA of RISC-V allows SoC developers to also build or modify a processor core optimized to the application requirements. The SoC verification tasks are adapting to address the significant increases in complexity. This article covers the 6 key components of RISC-V processor verification: The DV Plan, RTL DUT, Testbench, Tests, Reference model, and Siemens EDA Questa SystemVerilog simulation environment.
Within the RISC-V specification many standard extensions and options are available in addition to any user defined custom instructions. While some processor DV aspects may appear similar to a modern SoC verification flow, the flexibility of the open standard ISA of RISC-V makes almost every step uniquely challenging. This article provides some insights with the development of the latest architectural validation test suites for the RISC V Vectors draft specification, using Siemens EDA Questa with a UVM SystemVerilog testbench, including coverage analysis and results.
RISC-V processor verification is growing almost as fast as the adoption of RISC-V in SoC designs. This is due in part to the flexibility that is permitted with an open standard ISA (Instruction Set Architecture), which allows SoC developers to build or modify the processor design. As SoC developers address the additional processor verification tasks in the SoC design verification (DV) plans they are facing some significant increases in verification complexity. This article covers the 6 key components of RISC-V processor verification:
- The DV plan (including coverage metrics, debug modes and asynchronous events)
- RISC-V processor RTL implementation as the Device Under Test (DUT)
- SystemVerilog testbench IP with supporting infrastructure and interfaces
- Selection of test suites & generators, including directed, random, and compliance tests
- Reference model to compare against the same tests running on the DUT
- Siemens EDA Questa SystemVerilog simulation environment
Within the RISC-V specification many standard extensions and options are available in addition to any user defined custom instructions. While some processor DV aspects may appear similar to a modern SoC verification flow, the flexibility of the open standard ISA of RISC-V makes almost every step uniquely challenging. This article provides some insights with the development of the latest architectural validation test suites for the RISC-V Vectors draft specification, using Siemens EDA Questa with a UVM SystemVerilog testbench, including coverage analysis and results.
RISC-V PROCESSOR VERIFICATION
Processor verification can be significantly more complex than the SoC that is built around it. In the past SoC verification had a key assumption of ‘known good’ processor IP from a single source provider. The SoC verification was focused just on the interfaces and correct integration of the core IP within the design. Now with RISC-V all SoC developers have the flexibility to build a new core, use an open source or commercial core, and if necessary, add or extend a core with custom instructions. In addition to the design flexibility of RISC-V, adopters must also cover the resulting verification tasks. The flow diagram shown in Figure 1 illustrates a SystemVerilog testbench for RISC-V verification.
Figure 1 - RISC-V Verification Flow Based on SystemVerilog Testbench
RISC-V VERIFICATION: CONFIGURABLE REFERENCE MODEL
The RISC-V ISA is an open standard and the specifications are managed by RISC-V International and its working groups . The basic architecture is defined by the two base specifications known as unprivileged (user) and privileged mode. The specifications and optional standard extensions are shown in the table below, Figure 2.
Figure 2 - Table of RISC-V Specifications
It is typical for a hardware implementation to be developed around a number of these specifications and during the course of development adopt a number of the supported configurations and optional features. Since the specifications are all developed and maintained separately, it is to be expected that a design will be based on a cross section of the referenced versions. Indeed, during the development of the extensions, it is to be expected that lead developers will commit designs to silicon as the specification nears the ratification milestone. Therefore, for any verification plan the detailed specifications and all possible options need to be both documented and set-up correctly for the verification reference model.
The Imperas reference model covers the envelope of the specifications, and this can be configured to any of the valid optional features. Note that within each specification revision the type, number and description of these options can be amended, extended or otherwise altered. To be fully configurable, the reference model needs to have a multi-dimensional configuration selection for each extension specification version. Figure 2 shows the current version status at the time of writing (Feb 2021) for the ratified and near ratified specifications as supported by the Imperas reference model.
In addition to the standard extensions any reference model must support user defined extensions and custom instructions. However, since the model is still to be relied upon for all the standard extensions, the Imperas reference model supports custom instructions and registers in side file extensions, so as not to perturb the verified model.
RISC-V VERIFICATION: STEP-AND-COMPARE
In the verification flow outlined above the basic approach is to compare the same input stimulus to both the RTL (DUT) against the reference model. While this may be supported with a log file comparison approach, it does not accommodate the most sophisticated requirements around debug operations and asynchronous events. Figure 3 illustrates the approach known as the ‘step-and-compare’ method, which directly synchronizes the DUT and the reference model at every instruction retirement.
Figure 3 - SystemVerilog Encapsulated RISC-V Verification Reference Model
With the Imperas OVP reference model encapsulated within a SystemVerilog testbench, the support infrastructure and IP can be configured to run the RTL simulation in parallel to the reference model. This allows the same test sequences to run on the RTL and the reference model while maintaining synchronization. As an instruction retires, the full processor status can be compared to the reference model and thus identify any unexpected discrepancy with the granularity of the instruction boundary.
But detection of any issue is just the starting point for the analysis and correction process. By controlling the simulations and debug with a single user interface, it is possible to trace back the fully identify and explore the point of error or bug. Faults could arise from a number of situations, a bug in the RTL, a bug in the specification, a bug in the test bench, a bug in the test program, or a difference in the interpretation between the reference model and any of the above. Since RISC-V is an open ISA many developers could interpret the intention of the paper specification differently. Since the Imperas reference model has been subjected to extensive verification itself with many billions of instructions plus verified in test environments with almost all of the leading RISC-V implementations, it is considered a dependable reference. A recent example is a case study from the RISC-V Summit in December, which highlighted Seagate's production-quality RISC-V verification infrastructure based on the Imperas RISC-V Verification Reference Model.
RISC-V VERIFICATION: DEBUG
Having identified a bug or error in the RTL DUT, the verification task then becomes a detailed debug analysis to uncover the root cause. Since the Imperas reference model is encapsulated within the SystemVerilog testbench it permits a single simulation environment for both the RTL and the reference model. This allows a single debug session to control the operation of both the RTL DUT and the reference model. Since a discrepancy, bug or error may be related to a class of operations the full behavior can be explored and inspected. The efficient identification and resolution of bugs uncovered by test failures is one of the greatest benefits of the ‘Step-and-Compare’ methodology with the Imperas reference model, by identifying the earliest possible divergence. Another benefit is that a step and compare approach stops on first divergence. This is very different from a typical post simulation trace compare approach where often under error conditions the simulators can go ‘off into the weeds’ and thus waste time and compute resources.
RISC-V VERIFICATION: ASYNCHRONOUS EVENTS AND INTERRUPTS
Simulation is the foundation of all SoC (and processor IP) design and verification projects. Having configured the testbench and completed the basic test process with the key specification features of the processor, it now becomes necessary to simulate the full operational environment of a processor running actual software.
In normal operation a processor will be subject to a number of asynchronous events and interrupts due to both normal and also unusual situations. During the verification process the testbench can also be subjected to a collection of unusual, unexpected and distributive stimulus. Over and above the basic operation of the test suites it is important to test with multiple levels of cascading events. The full RISC-V privilege architecture including asynchronous interrupts, software exceptions, and full trap handlers need exploring. When the stimulus is obtained with a test generator the asynchronous events also need to be controlled so as to be both reproducible and recorded for coverage analysis.
RISC-V VERIFICATION: VECTOR EXTENSIONS
At the time of writing (Feb 2021) the RISC-V vector extensions are close to completing the ratification process. Having been in development by the RISC-V International Vectors working group the intention is to offer a broad range of configurable options and features. The RISC-V vector extensions are designed to support complex arithmetic operations required for applications involving linear algebra, such as supercomputers, cryptography, Artificial intelligence (AI), Machine Learning (ML) and Deep Learning (DL). A traditional or scalar ISA is based around operations on single data items, a Vector processor operates over an array of data items which enables hardware implementation / acceleration of key computational workloads.
The wide range of options and features defined within the Vectors extension is the most complex extension yet to be added to RISC-V. In fact, the complexity can be considered similar to all the other standard RISC-V extensions combined.
An architectural validation test suite is intended to provide some basic tests to ensure the implementation DUT is broadly in alignment with the overall high-level objectives of the specification. It is not a complete hardware verification test but it offers some confirmation of behavior from a programmer view of the expected processor operations.
The RISC-V Vector specification key definition parameters are xlen, elen, vlen, and slen. Imperas have released sample tests configured for RV32GCV with elen:32, vlen:256, slen:256 together with the free reference model known as riscvOVPsimPlus . The sample Imperas vector tests comprises 7 suites (vb, vf, vi, vm, vp, vr, and vx) of over 5,000 tests that cover the 300+ vector instructions with almost 3.5 million instructions tested.
By using the methodologies outlined in this article the architecture reference test for Vectors achieve coverage results of over 91% of the basic instruction functional coverage as shown in Figure 4.
Figure 4 - Table of Imperas RISC-V Test Suites and Coverage
RISC-V PROCESSOR VERIFICATION: Why Not Also Develop Software Early?
The RISC-V processor verification tasks differ from a traditional SoC verification in a number of ways, most noticeably by the complexity added when running software on the processor. Indeed, the best way to verify the full and correct operation of a processor is to run test cases based on the instruction streams (software) around key test case scenarios. The interaction of software across the system with asynchronous events and interrupts is also an import system level design issue.
The Imperas reference model, shown above encapsulated with a SystemVerilog testbench for hardware RTL verification, can also be used independently. Having developed the configuration setting for the hardware DV tasks, it is also possible to run the platform independently as a software development target. By adding and extending the processor model with key peripherals the virtual platform can be used to develop firmware and OS drivers. The Imperas commercial VAP (Verification, Analysis, and Profiling) tools and simulation technology  support the open source library of processor model and example platforms available on the OVP (Open Virtual Platforms) website . Developing the software early also helps the SoC system level design with real applications and workloads intended for the final device. The virtual platform offers system wide debug and analysis with heterogeneous support for multicore designs in a single debug and analysis environment. This offers unique introspection that helps not just to develop early drivers and firmware, but throughout the software lifecycle.
In the past SoC verification flows had a fundamental assumption based on ‘known good’ processor IP from a single source supplier. By definition SoC verification was previously only concerned with testing the interfaces between the processor IP and the rest of the SoC, i.e., just a basic IP integration test. Now with RISC-V all SoC developers have the flexibility to build a new core, use an open source or commercial core, and if required, add or extend a core with custom instructions. With this flexibility, the SoC developers must also accept the additional verification challenges.
Thus, the SoC verification flow needs to adapt and cover the new flexibility that RISC-V offers. Since SystemVerilog and UVM are the established methods for SoC verification, the use of the encapsulated Imperas silicon-proven golden reference model helps the transition to effective RISC-V processor verification. By using the ‘step-and-compare’ methodology with a UVM SystemVerilog testbench, Imperas golden reference model, and Siemens EDA Questa, the RISC-V verification flow supports both coverage and asynchronous events.
The authors wish to thank the following RISC-V developers that use riscvOVPsim, riscvOVPsimPlus and Imperas commercial simulators that depend on the Imperas Reference Model, including: Mellanox/Nvidia, Seagate, NSITEXE/Denso, Google Cloud, Chips Alliance, lowRISC, OpenHW Group, Andes, Valtrix, Nagra/Kudelski, Silicon Labs, Incore Semi, RISC-V Compliance Working Group, Symbiotic EDA, Thales, Hensoldt Cyber, Invia, and the all others yet to be announced publicly, plus all the support the Imperas team received under the Siemens EDA Questa Vanguard program.
Imperas is the leading provider of RISC-V processor models, hardware design verification solutions, and virtual prototypes for software simulation. Imperas, along with Open Virtual Platforms (OVP), promotes open source model availability for a spectrum of processors, IP vendors, CPU architectures, system IP and reference platform models of processors and systems ranging from simple single core bare metal platforms to full heterogeneous multi-core systems booting SMP Linux. All models are available from Imperas at www.imperas.com and the Open Virtual Platforms (OVP) website at www.ovpworld.org.
- RISC-V International ISA specification
- riscvOVPsimPlus available for free
- Imperas commercial tools and solutions
- OVP (Open Virtual Platforms) processor models
Back to Top