Verification Academy

Search form

My Account Menu

  • Register
  • Log In
  • Topics
  • Courses
  • Forums
  • Patterns Library
  • Cookbooks
  • Events
  • More
  • All Topics
    The Verification Academy offers users multiple entry points to find the information they need. One of these entry points is through Topic collections. These topics are industry standards that all design and verification engineers should recognize. While we continue to add new topics, users are encourage to further refine collection information to meet their specific interests.
    • Languages & Standards

      • Portable Test and Stimulus
      • Functional Safety
      • Design & Verification Languages
    • Methodologies

      • UVM - Universal Verification Methodology
      • UVM Framework
      • UVM Connect
      • FPGA Verification
      • Coverage
    • Techniques & Tools

      • Verification IP
      • Simulation-Based Techniques
      • Planning, Measurement, and Analysis
      • Formal-Based Techniques
      • Debug
      • Clock-Domain Crossing
      • Acceleration
  • All Courses
    The Verification Academy is organized into a collection of free online courses, focusing on various key aspects of advanced functional verification. Each course consists of multiple sessions—allowing the participant to pick and choose specific topics of interest, as well as revisit any specific topics for future reference. After completing a specific course, the participant should be armed with enough knowledge to then understand the necessary steps required for maturing their own organization’s skills and infrastructure on the specific topic of interest. The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization’s processes so that you can then reap the benefits that advanced functional verification offers.
    • Universal Verification Methodology (UVM)

      • Advanced UVM
      • Basic UVM
      • Introduction to UVM
      • UVM Connect
      • UVM Debug
      • UVMF - One Bite at a Time
    • Featured Courses

      • Introduction to ISO 26262
      • Introduction to DO-254
      • Clock-Domain Crossing Verification
      • Portable Stimulus Basics
      • Power Aware CDC Verification
      • Power Aware Verification
      • SystemVerilog OOP for UVM Verification
    • Additional Courses

      • Assertion-Based Verification
      • An Introduction to Unit Testing with SVUnit
      • Evolving FPGA Verification Capabilities
      • Metrics in SoC Verification
      • SystemVerilog Testbench Acceleration
      • Testbench Co-Emulation: SystemC & TLM-2.0
      • Verification Planning and Management
      • VHDL-2008 Why It Matters
    • Formal-Based Techniques

      • Formal Assertion-Based Verification
      • Formal-Based Technology: Automatic Formal Solutions
      • Formal Coverage
      • Getting Started with Formal-Based Technology
      • Handling Inconclusive Assertions in Formal Verification
      • Sequential Logic Equivalence Checking
    • Analog/Mixed Signal

      • AMS Design Configuration Schemes
      • Improve AMS Verification Performance
      • Improve AMS Verification Quality
  • All Forum Topics
    The Verification Community is eager to answer your UVM, SystemVerilog and Coverage related questions. We encourage you to take an active role in the Forums by answering and commenting to any questions that you are able to.
    • UVM Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • UVM Forum
    • SystemVerilog Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • SystemVerilog Forum
    • Coverage Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • Coverage Forum
    • Additional Forums

      • Announcements
      • Downloads
      • OVM Forum
  • Patterns Library
    The Verification Academy Patterns Library contains a collection of solutions to many of today's verification problems. The patterns contained in the library span across the entire domain of verification (i.e., from specification to methodology to implementation—and across multiple verification engines such as formal, simulation, and emulation).
    • Implementation Patterns

      • Environment Patterns
      • Stimulus Patterns
      • Analysis Patterns
      • All Implementation Patterns
    • Specification Patterns

      • Occurrence Property Patterns
      • Order Property Patterns
      • All Specification Patterns
    • Pattern Resources

      • Start Here - Patterns Library Overview
      • Whitepaper - Taking Reuse to the Next Level
      • Verification Horizons - The Verification Academy Patterns Library
      • Contribute a Pattern to the Library
  • All Cookbooks
    Find all the methodology you need in this comprehensive and vast collection. The UVM and Coverage Cookbooks contain dozens of informative, executable articles covering all aspects of UVM and Coverage.
    • UVM Cookbook

      • UVM Basics
      • Testbench Architecture
      • DUT-Testbench Connections
      • Configuring a Test Environment
      • Analysis Components & Techniques
      • End Of Test Mechanisms
      • Sequences
      • The UVM Messaging System
      • Other Stimulus Techniques
      • Register Abstraction Layer
      • Testbench Acceleration through Co-Emulation
      • Debug of SV and UVM
      • UVM Connect - SV-SystemC interoperability
      • UVM Versions and Compatibility
      • UVM Cookbook
    • Coding Guidelines & Deployment

      • Code Examples
      • UVM Verification Component
      • Package/Organization
      • Questa/Compiling UVM
      • SystemVerilog Guidelines
      • SystemVerilog Performance Guidelines
      • UVM Guidelines
      • UVM Performance Guidelines
    • Coverage Cookbook

      • Introduction
      • What is Coverage?
      • Kinds of Coverage
      • Specification to Testplan
      • Testplan to Functional Coverage
      • Bus Protocol Coverage
      • Block Level Coverage
      • Datapath Coverage
      • SoC Coverage Example
      • Requirements Writing Guidelines
      • Coverage Cookbook
  • All Events
    No one argues that the challenges of verification are growing exponentially. What is needed to meet these challenges are tools, methodologies and processes that can help you transform your verification environment. These recorded seminars from Verification Academy trainers and users provide examples for adoption of new technologies and how to evolve your verification process.
    • Upcoming & Featured Events

      • U2U MARLUG - January 26th
      • Creating an Optimal Safety Architecture  - February 9th
      • The ABC of Formal Verification - February 11th
      • Events Calendar
    • On Demand Seminars

      • I'm Excited About Formal...
      • Visualizer Coverage
      • Formal-based ‘X’ Verification
      • 2020 Functional Verification Study
      • All On-Demand Seminars
    • Recording Archive

      • Improving Your SystemVerilog & UVM Skills
      • Should I Kill My Formal Run?
      • Visualizer Debug Environment
      • All Recordings
    • Mentor Training Center

      • SystemVerilog for Verification
      • SystemVerilog UVM
      • UVM Framework
      • Instructor-led Training
    • Mentor Learning Center

      • SystemVerilog Fundamentals
      • SystemVerilog UVM
      • Questa Simulation Coverage Acceleration Apps with inFact
      • View all Learning Paths
  • About Verification Academy
    The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization's processes so that you can then reap the benefits that advanced functional verification offers.
    • Blog & News

      • Verification Horizons Blog
      • Academy News
      • Academy Newsletter
      • Technical Resources
    • Verification Horizons Publication

      • Verification Horizons - November 2020
      • Verification Horizons - July 2020
      • Verification Horizons - March 2020
      • Issue Archive
    • About Us

      • Verification Academy Overview
      • Subject Matter Experts
      • Contact Us
    • Training

      • Questa® & ModelSim®
      • Questa® inFact
      • Functional Verification Library
  • Home /
  • Verification Horizons /
  • March 2020 /
  • Verify Thy Verifyer

Verify Thy Verifyer

Verification Horizons - Tom Fitzpatrick, Editor

 | Verification Horizons - March 2020 by Srinivasan Venkataramanan and Ajeetha Kumari, VerifWorks

INTRODUCTION

Design Verification is a field that requires a lot of thinking and equally a lot of coding. Tighter time-to-market adds a lot of schedule pressure to the teams coding those testbenches and test cases.

The advent of UVM (Universal Verification Methodology) as the standard framework, has helped the industry make good progress in terms of structured testbenches. One of the primary objectives of UVM is to build robust, reusable testbenches. UVM has a set of guidelines on how to build reusable components and also provides a Base Class Library (BCL) that implements basic infrastructure.

As with any code development, verifying the code for compliance, quality and of-course functionality is a tedious task, but a much needed one. In this article we share our experience in helping customers use an innovative technology named Karuta to perform rule-checking on the UVM code base.

EXPECTATIONS FROM A GOOD UVM-BASED TESTBENCH

With UVM now a mature technology, there are several expectations on teams using UVM from its benefits standpoint.

Management Expectations on UVM

Management expects high quality code that can be reused across generations of products. Time-to-market (TTM) is a key metric that management tracks on every project. Another key management concern is the cost of developing and maintaining the UVM code base especially the lower level code. The cost factor drives much of the code to be delegated to junior engineers.

Technical Leads' Expectations on UVM

As with any large framework, there is more than one way to get things done in UVM. As a technical lead, one wants to establish a set of coding guidelines/best practices and ensure that his/her team members adhere to the same. Typical concerns from a technical lead perspective are:

  • Simulator performance – impacted if one makes poor choices in UVM
  • Debug-ability – use consistent debug hooks to improve turn-around time
  • Productivity BKMs (Best Known Methods)

Leads would also benefit from an automated audit of UVM code prior to every milestone release of their teams’ code base.

Figure 1 - UVM testbench expectations


Verification Engineers’ Expectations on UVM

Not every verification engineer is lucky enough to have been trained by UVM experts. Given the myriad of base classes and features such as TLM, Factory, Config-DB, junior engineers find it hard to comprehend at the beginning. Also the ability to get things done in many ways in UVM adds to their chaos. Typical expectations from a verification engineer would be:

  • A concise set of dos and don’ts (aka rules)
  • A brief description of why such rules exist
  • How to ensure his/her code is compliant to a basic set of key rules, say during every check-in to a revision control mechanism (Git/SVN/P4/SOS/Clearcase, etc.)
  • Handy debug tricks that can save hours for him/her during the coding process

WHAT CAN GO WRONG WITH UVM-BASED TESTBENCHES?

Depending on the team’s expertise answer to this question would vary; however on an average, we have observed that a typical UVM testbench provides about 20-30% scope for improvement. The specific area of focus (we refer to them as perspect-ives) can vary across teams, typical ones being:

  • Performance
  • Reuse
  • Debug
  • Functionality (of TB typically, at times design as well, albeit indirectly)
  • Productivity
  • Coding styles (aka Lint)

Over the past few years, attempts have been made to apply the concept of Linting to testbenches. Given the success of RTL-lint tools, this is a no-brainer attempt. However, verification engineers are smart and are on a tight schedule by itself. Linting usually throws a sea of messages and finding perspectives such as the ones listed above is nothing less than finding a needle in a haystack. Attempting to perform a “lint” type of checks on a UVM code base simply falls flat and would discourage adoption of such technology.

Given that verification as a discipline employs a lot of junior engineers, who at times do not understand the intricacies of a complex methodology such as UVM leads to sub-optimal code.

SAMPLE RULES, CODE SNIPPETS AND POTENTIAL FIXES

Having looked at what can go wrong with UVM code, let’s look at some real-life use cases.

Performance Perspective - Example

UVM supports wildcard usage in uvm_config_db::set/get API. Though it is at times handy to use, this comes with a performance penalty as the search for wildcards goes through DPI. Also using a single wildcard anywhere in inst_name incurs a significant penalty in the UVM search algorithm, as can be seen in the code comments from the UVM BCL in Figure 2, below.

Figure 2 - UVM BCL code comments


The snippet is extracted from the UVM 1.1d base class and is an indicative of what a user's poor code can lead to. Even a single use of the wildcard in inst_name would enable a “greedy” search algorithm for all upcoming uvm_config_db searches. Below is an example of user code demonstrating a poor style:

uvm_config_db::set(.cntxt(this), .inst_name (“axi_ag_0”),
	               .field_name(“*enable*”), .value(1) );

Potential fixes for the above style, with better per-formance, same functionality with little verbose code is shown below:

1. ENABLE SCOREBOARD_0:

uvm_config_db::set(.cntxt(this), .inst_name (“axi_ag_0”),
                       .field_name(“sbrd_enable_0”), .value(1) );

2. ENABLE SCOREBOARD_1:

uvm_config_db::set(.cntxt(this), .inst_name (“axi_ag_0”),
                       .field_name(“sbrd_enable_1”), .value(1) );

3. ENABLE FUNCTIONAL COVERAGE:

uvm_config_db::set(.cntxt(this), .inst_name (“axi_ag_0”),
                       .field_name(“fcov_enable”), .value(1) );

One of the challenges is to identify a list of field names that an existing (poor style) wildcard expands to.

Debug Perspective - Example

UVM has built-in debug hooks for easier log file based analysis. Problem is, engineers are not always aware of these hooks or they do not add these in production environments, leading to costly debug cycles. We show two such features below:

1. UVM FACTORY PRINT

function void axi_env::start_of_simulation_phase(uvm_phase phase);
 uvm_factory f_h;
 super.start_of_simulation_phase (.phase(phase));
 f_h = uvm_factory::get();
 f_h.print(.all_types(1));
endfunction : start_of_simulation_phase

A sample log from Questa® with the above debug hook is shown below.


This guideline/rule saying “factory.print()” shall be invoked at the right time/phase in a simulation goes a long way in debug as this information appears early in simulation (usually 0 time), hence one can be sure whether the intended overrides worked as per the test specification/requirements.

2. UVM TOPOLOGY PRINT

UVM components are built hierarchically and at time 0. It is essential to get the testbench configured properly before running simulations and debugging the design behavior. One of the very useful debug hooks in UVM is the popular print_topology() – though popular, it is not called by default by UVM. Engineers also do not always remember to call this API leading to unnecessary debug cycles.

In Figure 3, below, is a sample output from this debug API.

Figure 3 - Debug API sample output


A code snippet enabling the print output is shown below:

function void axi_env::end_of_elaboration_phase(uvm_phase phase);
 uvm_top t_h;
 super. end_of_elaboration_phase (.phase(phase));
 t_h = uvm_top::get();
 t_h.print_topology();
endfunction : end_of_elaboration_phase

Reuse Perspective - Example

1. USING FACTORY BASED CREATION

One of the primary motivations to use UVM in a pro-ject is to keep various components and transactions reusable; this is achieved by deferring the creation of objects to a factory based approach instead of direct new() invocation. A key check will be to ensure all interesting objects are indeed following this rule.

The code snippet below demonstrates this key rule.

function void axi_env::build_phase (uvm_phase phase);
 super. build _phase (.phase(phase));
 this.axi_agt_0 = axi_agent::type_id::create(.name(“axi_agt_0”), .parent(this));
 
 // POOR code this.axi_agt_0 = new (.name(“axi_agt_0”), .parent(this));
endfunction : build_phase

VERIFYING THE VERIFYER

So how do we “verify” the verification code itself? There are at least two angles to this question:

  1. Purely functional correctness
  2. Quality perspectives such as:
  3. a. Debug-ability of the code (Directly impacts schedule)
    b. Simulation Performance friendly code (Impacts licensing costs and time)
    c. Reusability perspective (Impacts the time for the next task/project), etc.

With domain experts and using the right off-the-shelf VIPs such as Questa® VIPs, the functionality angle can be addressed. What about the second angle?

We at VerifWorks have done years of consulting and have learned several best practices and have encountered poor coding styles. Over the years, we have accumulated these findings to a new solution named Karuta (Design Verification Rule checker). When run on a typical UVM- based testbench along with Questa®, Karuta highlights various strengths and weaknesses of users’ UVM code thereby helping them address the quality angle.

We ran Karuta on some of the popular, open source, non-trivial UVM code bases from Google, Cadence, Juniper, Ariane RISC–V®, TVIP and more. Our analysis into these code bases reveal interesting results, that left us baffled. Below is a sneak preview in Table 1.

Table 1 - Code base analysis


Do visit us at the upcoming DVCon US 2020 exhibition to learn more about these results and how Karuta can help in your projects.

Back to Top

Table of Contents

Verification Horizons Articles:

  • Teams that Share the Credit More Often Share Success

  • Verify Thy Verifyer

  • Using Questa® SLEC to Speed Up Verification of Multiple HDL Outputs

  • AI-Based Sequence Detection

  • An Open Data Management Tool for Design and Verification

  • Detecting Security Vulnerabilities in a RISC-V® Based System-on-Chip

  • Formal Verification of RISC-V® Processors

© Mentor, a Siemens Business, All rights reserved www.mentor.com

Footer Menu

  • Sitemap
  • Terms & Conditions
  • Verification Horizons Blog
  • LinkedIn Group
SiteLock