Verification Academy

Search form

My Account Menu

  • Register
  • Log In
  • Topics
  • Courses
  • Forums
  • Patterns Library
  • Cookbooks
  • Events
  • More
  • All Topics
    The Verification Academy offers users multiple entry points to find the information they need. One of these entry points is through Topic collections. These topics are industry standards that all design and verification engineers should recognize. While we continue to add new topics, users are encourage to further refine collection information to meet their specific interests.
    • Languages & Standards

      • Portable Test and Stimulus
      • Functional Safety
      • Design & Verification Languages
    • Methodologies

      • UVM - Universal Verification Methodology
      • UVM Framework
      • UVM Connect
      • FPGA Verification
      • Coverage
    • Techniques & Tools

      • Verification IP
      • Simulation-Based Techniques
      • Planning, Measurement, and Analysis
      • Formal-Based Techniques
      • Debug
      • Clock-Domain Crossing
      • Acceleration
  • All Courses
    The Verification Academy is organized into a collection of free online courses, focusing on various key aspects of advanced functional verification. Each course consists of multiple sessions—allowing the participant to pick and choose specific topics of interest, as well as revisit any specific topics for future reference. After completing a specific course, the participant should be armed with enough knowledge to then understand the necessary steps required for maturing their own organization’s skills and infrastructure on the specific topic of interest. The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization’s processes so that you can then reap the benefits that advanced functional verification offers.
    • Universal Verification Methodology (UVM)

      • Advanced UVM
      • Basic UVM
      • Introduction to UVM
      • UVM Connect
      • UVM Debug
      • UVMF - One Bite at a Time
    • Featured Courses

      • Introduction to ISO 26262
      • Introduction to DO-254
      • Clock-Domain Crossing Verification
      • Portable Stimulus Basics
      • Power Aware CDC Verification
      • Power Aware Verification
      • SystemVerilog OOP for UVM Verification
    • Additional Courses

      • Assertion-Based Verification
      • An Introduction to Unit Testing with SVUnit
      • Evolving FPGA Verification Capabilities
      • Metrics in SoC Verification
      • SystemVerilog Testbench Acceleration
      • Testbench Co-Emulation: SystemC & TLM-2.0
      • Verification Planning and Management
      • VHDL-2008 Why It Matters
    • Formal-Based Techniques

      • Formal Assertion-Based Verification
      • Formal-Based Technology: Automatic Formal Solutions
      • Formal Coverage
      • Getting Started with Formal-Based Technology
      • Handling Inconclusive Assertions in Formal Verification
      • Sequential Logic Equivalence Checking
    • Analog/Mixed Signal

      • AMS Design Configuration Schemes
      • Improve AMS Verification Performance
      • Improve AMS Verification Quality
  • All Forum Topics
    The Verification Community is eager to answer your UVM, SystemVerilog and Coverage related questions. We encourage you to take an active role in the Forums by answering and commenting to any questions that you are able to.
    • UVM Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • UVM Forum
    • SystemVerilog Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • SystemVerilog Forum
    • Coverage Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • Coverage Forum
    • Additional Forums

      • Announcements
      • Downloads
      • OVM Forum
  • Patterns Library
    The Verification Academy Patterns Library contains a collection of solutions to many of today's verification problems. The patterns contained in the library span across the entire domain of verification (i.e., from specification to methodology to implementation—and across multiple verification engines such as formal, simulation, and emulation).
    • Implementation Patterns

      • Environment Patterns
      • Stimulus Patterns
      • Analysis Patterns
      • All Implementation Patterns
    • Specification Patterns

      • Occurrence Property Patterns
      • Order Property Patterns
      • All Specification Patterns
    • Pattern Resources

      • Start Here - Patterns Library Overview
      • Whitepaper - Taking Reuse to the Next Level
      • Verification Horizons - The Verification Academy Patterns Library
      • Contribute a Pattern to the Library
  • All Cookbooks
    Find all the methodology you need in this comprehensive and vast collection. The UVM and Coverage Cookbooks contain dozens of informative, executable articles covering all aspects of UVM and Coverage.
    • UVM Cookbook

      • UVM Basics
      • Testbench Architecture
      • DUT-Testbench Connections
      • Configuring a Test Environment
      • Analysis Components & Techniques
      • End Of Test Mechanisms
      • Sequences
      • The UVM Messaging System
      • Other Stimulus Techniques
      • Register Abstraction Layer
      • Testbench Acceleration through Co-Emulation
      • Debug of SV and UVM
      • UVM Connect - SV-SystemC interoperability
      • UVM Versions and Compatibility
      • UVM Cookbook
    • Coding Guidelines & Deployment

      • Code Examples
      • UVM Verification Component
      • Package/Organization
      • Questa/Compiling UVM
      • SystemVerilog Guidelines
      • SystemVerilog Performance Guidelines
      • UVM Guidelines
      • UVM Performance Guidelines
    • Coverage Cookbook

      • Introduction
      • What is Coverage?
      • Kinds of Coverage
      • Specification to Testplan
      • Testplan to Functional Coverage
      • Bus Protocol Coverage
      • Block Level Coverage
      • Datapath Coverage
      • SoC Coverage Example
      • Requirements Writing Guidelines
      • Coverage Cookbook
  • All Events
    No one argues that the challenges of verification are growing exponentially. What is needed to meet these challenges are tools, methodologies and processes that can help you transform your verification environment. These recorded seminars from Verification Academy trainers and users provide examples for adoption of new technologies and how to evolve your verification process.
    • Upcoming & Featured Events

      • CDC+RDC Analysis - 4/20
      • Low Power Verification - 4/29
      • Fault Campaign for Mixed-Signal - 5/4
      • User2User - 5/26
      • Webinar Calendar
    • On-Demand Webinars

      • Basic Abstraction Techniques
      • Safety Analysis Techniques
      • QVIP Workflow and Debug for PCIe
      • Writing a Proxy-driven Testbench
      • Achieving High Defect Coverage
      • Visualizer Features
      • Questa Static and Formal Apps
      • All On-Demand Webinars
    • Recording Archive

      • Siemens EDA 2021 Functional Verification Webinar Series
      • Improving Your SystemVerilog & UVM Skills
      • Should I Kill My Formal Run?
      • Visualizer Debug Environment
      • Industry Data & Surveys
      • All Recordings
    • Conferences

      • DVCon 2021
      • DVCon 2020
      • DAC 2019
      • All Conferences
    • Mentor Learning Center

      • SystemVerilog Fundamentals
      • SystemVerilog UVM
      • View all Learning Paths
  • About Verification Academy
    The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization's processes so that you can then reap the benefits that advanced functional verification offers.
    • Blog & News

      • Verification Horizons Blog
      • Academy News
      • Academy Newsletter
      • Technical Resources
    • Verification Horizons Publication

      • Verification Horizons - March 2021
      • Verification Horizons - November 2020
      • Verification Horizons - July 2020
      • Issue Archive
    • About Us

      • Verification Academy Overview
      • Subject Matter Experts
      • Contact Us
    • Training

      • Questa Basic
      • Questa Advanced
      • Mastering Questa
  • Home
  • Verification Horizons
  • June 2013
  • Non-invasive Software Verification Using Vista Virtual Platforms

Non-invasive Software Verification Using Vista Virtual Platforms

Verification Horizons - Tom Fitzpatrick, Editor

Non-invasive Software Verification using Vista Virtual Platforms by Alex Rozenman, Vladimir Pilko, and Nilay Mitash, Mentor Graphics

Introduction

With the SoCs now supporting Multi-Core processors, complex ASICs and combinations that include systems on a board, SoC implementations now become an ever growing challenge for software development. Software development has to be supported not only by the inclusion of an RTOS, but, many SoC providers now have to leverage upon the Bare-Metal concept to achieve the necessary demands of today's SoCs. However, there still exists a huge chasm in the software development arena that addresses both the need to be able to verify not only the sw/hw interactions, but, also the software itself in a hardware context. This has become almost a necessity in today's "security" based systems marketplace.

As large and complex SoCs trend deeper into embedded environments, they need to be seamlessly implemented and contained, for verification of both software and hardware in the same context. The following sections cover the underlying technologies used in context to each of the areas and explain the mechanisms for achieving verification of the software itself.

What Is Aspect Oriented Software Verification?

In almost every intelligent applicable software function, there are always locations that have meaningful implications. A reflective functional equivalent of these locations would create the verification logic. For example, an assertion in the code can be reflectively mapped to its verification logic, making sure that the conditions that cause the assertion to fire are captured in the verification logic. Each such verification logic is strongly anchored to its aspect in the original software.

From AOP (Aspect Oriented Programming) point of view, the verification locations can be seen as "join points", verification logic as a set of "advices" and the verification script as the AOP program itself.

The following code locations/anchors that are found in almost all software code could be used:

  • Absolute execution addresses (PC)
  • Source locations (file:line)
  • Function entry/exit
  • State change points (watch points)
  • Peripheral device accesses and programming
  • Interrupts
  • CPU exceptions

In typical software, verification logic may include assertions, invariant conditions checkers, print outs for flow tests, fault injection or an external stimulus. We will use the term "verification script" to denote a program that attaches the verification logic to its corresponding locations.

The meta-information generated from a standard compilation of the target software is used by the verification scripts. The verification scripts commonly have access to the meta-information. The meta-information access allows it to explore the software structure and attach advices (verification logic) to the selected locations.

Differentiating Aspect-oriented Software Verification From Aop In Hardware Verification Environments

Using an aspect-oriented approach to verifying software allows the injection of additional verification code without altering the basic execution of the software itself, from the perspective of the system. As such, this approach has the advantage of being orthogonal to the functionality being verified. This is different from the typical view of hardware verification, where we build a testbench to model the environment in which the hardware will operate.

One value in using an approach like UVM for developing a verification environment for hardware is that you can use object-oriented techniques to modify the functionality and structure of your testbench without having to rewrite your code. Using AOP in this context is not recommended, nor required, because the injected code directly affects the functionality of testbench without providing sufficient visibility to the user.

Enabling Non-intrusive Jit Based Aspect Oriented Verification Environment

In a QEMU based processor modeling technology (ISS) based on JIT (Just-in-Time) compilation infrastructure, the JIT compiler is a binary translation tool, which is able to convert target binary code (e.g. ARM) into a host binary code (x86). The conversion is performed block by block and on demand. When the target execution reaches an instruction for the first time, only then, a short sequence of instructions is translated to the host binary code. The size of the compilation depends on many different factors, for example, blocks cannot include jump instruction, except, only at the end.

Running target software under JIT based processor model allows injecting verification logic non-intrusively and exclusively into the simulator context. The non-intrusive injection implies that verification logic does not affect any aspect of target behavior, target timing and power. The execution order of the target code is also left unmodified even when modeled with multiple threads or multi-core execution. Target debug information (DWARF: a widely used, standardized debugging data format generated by software compilers) is used in order to explore the software structure and provide reflection information to the verification scripts.

The aspect oriented JIT compiler provides a non-intrusive way of creating various test enablers. Some of the direct advantages are the capability to use this technology for tracing function call, enabling specific profiling, targeting coverage, and, inserting tracepoints (unlike breakpoints, tracepoints are software 'triggers' that do not stop the simulation). Tracepoints can be used to inject verification logic into arbitrary places in the target software. By accessing the DWARF debug data that contains information on the software structure, types, objects and source locations, the source code lines and function names can be converted into the needed addresses. A small "C" compiler is used to capture the tracepoint actions specified by "C" programs.

Applying The Aop Verification As An Advanced Software Unit Test Environment

As the software content in systems grows, more often than not, it is the software components itself that need to be verified while in in-system execution, i.e., without disturbing the overall functionality. Software based aspect oriented verification capabilities best suit this need to create a unit test environment.

Typical software unit test environment requires the ability to execute specific functions in specific predictable environments, be able to randomize arguments and state, collect execution results, compare them with a predefined gold values, and, all the while collect code coverage during the entire process. There is a very big intersection between this requirement and the abilities provided by a JIT based aspect oriented verification environment. The JIT based aspect oriented verification methodology is then able to initiate the needed unit test flow by injecting the needed advices and the verification logic that entirely replace the normal execution flow. Software coverage is almost a by-product of this methodology.

Figure 1: This shows a snapshot of all the function calls and their time duration in the Mentor software profiler

 This shows a snapshot of all the function calls and their time duration in the Mentor software profiler.


Using Jit And Aop For Software Trace And Analysis

Graphical viewers and visualization is a powerful tool for better understanding of software execution flow in a greater level of detail. JIT based aspect oriented verification environments allow to trace many kinds of software and hardware events non-intrusively without any need in code instrumentation. The following capabilities can be very efficiently traced:

  • Function entry and exit to show the call graph
  • CPU state (showing IDLE/BUSY/exceptions)
  • Caches/Snoops state and activity
  • Peripheral state and activity

Graphical analysis of both hardware and software together provides broad opportunities to find bottle necks, performance and power issues in a complex SoC system. Cache and snoop analysis can identify cache ineffective code fragments, false sharing and even hardto- debug software bug related to cache control. CPU state visualization together with function calls can help to identify interrupt latency bottle necks, unneeded preemption and dead locks in the code. Peripheral tracing may help to debug driver code, analyze bus contention and power issues.

As the simulation proceeds, the captured data can be displayed in a graphical format that shows the timelines of execution and their results. In Figure 1, the top portion (Function Calls) shows the calls to various functions as different compiled instructions for the CPU are executed. The state of the CPU is also captured, to show that the CPU is actually BUSY or IDLE during that time, shown by header CORE_STATE. The ISR plot shows when the Interrupt Service Routine triggered along with the SysTick provided by the ARM SystemC Fast Model. These plots enable intelligent debug of the software execution on the CPU.

Using Jit And Aop For Software Function Profiling

Another very important technique used in software verification is profiling. Profiling may be performed with respect to time, cache-hit, cache-miss, software-hardware execution and power. Software function profiling constitutes of tracing/profiling of instructions across single functions, functions stacks, and stack fragments.

A typical profiling analysis report contains details of functions self-time (i.e. amount of time consumed by instructions which belongs to the function itself) and functions total-time (i.e. actual duration of the function execution from entry till exit). This type of profiling is slightly enhanced, since it not only covers the execution of the function in the CPU context, but, also with respect to the effects of the instruction to and its return from an I/O.

Profiling enables the software developer to visualize the round-trip and nested function calls, and, its effects of a particular instruction executed on the target. Other traditional areas of software profiling, for example, Multi- Core processor execution, time and overlap of shared functions, are still available, and remain software centric.

JIT based aspect oriented verification environment allows to perform the software profiling completely non-intrusively requiring very small attention from the user. All the needed information can be automatically extracted from the meta-data available for the verification script. Ability to perform function profiling without explicit recompilation that also keeps the software semantics unmodified becomes a unique feature available for customers.

This plot clearly shows when a main_loop call made recursive calls, for example, to cs3_isr_ systick, which in turn calls udivdi3 functions, and when they return. After the final call to write happened, the recursive call ended and returned to the main_loop function.

Figure 2: A detailed analysis profiled for nested function calls.

 A detailed analysis profiled for nested function calls.


Enabling Software Coverage

Coverage in Software has been around for many years. Software code coverage is used to determine the quality of test-benches, flow tests, unit tests and the tested code itself. In addition it can be used to highlight "dead code", a major requirement in security based verification. The following types of coverage are well known:

  • Instruction: Checks if instructions were executed
  • Branch: Checks if conditional branches were both taken and not taken
  • Block: Checks if lexical blocks were executed
  • Path: Checks if all possible block combinations are executed
  • Logical expression: Check if all possible combinations of logical conditions were executed

Simple evaluation of structural coverage is not sufficient. To this effect, coverage also needs to include MC/DC (modified condition/decision coverage). MC/DC will provide an additional means of verifying software coverage in security sensitive applications.

All coverage information can be collected and assimilated with the hardware coverage information, thus, providing a comprehensive coverage of the entire sw/hw system.

Conclusion

In today's verification environment, software verification still lags behind the hardware verification. Hardware verification has a proven and beaten path pervasively adopted in the form of UVM.

Software verification tends to be a collection of in-house and vendor tools, with all its intellectual property focused on the creativity of Software Validation Engineers. However, software verification is quickly becoming one of the biggest challenges for complex ASIC platforms, SoCs and systems. In many cases, the SoCs and systems platforms are rated as life-threatening, such as medical devices, lethal and destructive, such as smart armaments, affect our daily lives, such as automotive and transportation, or data handlers, such as networking applications. In all of these, software is quietly becoming a dominant design-in and reliability factor. To set right a misalignment of directional vector, after deployment, of say a satellite in space, the only means available may be a pure software intervention.

It is important to note, that, the article has attempted to address the need for software verification by taking the route of aspect oriented methodology, but, clearly differentiates the needs for hardware verification from this approach. The objective of this article is to enable the flexibility of software aspect oriented verification for complex platforms that get affected by any intrusive functions or logic, and can lead to fatal exceptions in normal software execution.

Mentor Vista tools have taken a bold step in understanding these requirements and providing tools and methodologies that will enable a software developer to release products with confidence and assurance.

References:

  1. Aspect oriented Programming: https://en.wikipedia.org/wiki/Aspect-oriented_programming
  2. Mentor Vista tools
  3. Mentor Embedded CodeSourcery tools

Back to Top

Table of Contents

Verification Horizons Articles:

  • An Up-sized DAC Issue Takes the Stage

  • Interviewing a Verification Engineer

  • Maximum Productivity with Verification IP

  • Power Up Hardware/Software Verification Productivity

  • Non-invasive Software Verification using Vista Virtual Platforms

  • QVM: Enabling Organized, Predictable, and Faster Verification Closure

  • Verifying High Speed Peripheral IPs

  • Confidence in the Face of the Unknown: X-state Verification

  • Making it Easy to Deploy the UVM

  • NoC Generic Scoreboard VIP

  • Flexible UVM Components: Configuring Bus Functional Models

  • Monitors, Monitors Everywhere – Who Is Monitoring the Monitors?

  • The Need for Speed: Understanding Design Factors that Make Multi-core Parallel Simulations Efficient

Siemens Digital Industries Software

Siemens Digital Industries Software

##TodayMeetsTomorrow

Solutions

  • Cloud
  • Mendix
  • Siemens EDA
  • MindSphere
  • Siemens PLM
  • View all portfolio

Explore

  • Digital Journeys
  • Community
  • Blog
  • Online Store

Siemens

  • About Us
  • Careers
  • Events
  • News and Press
  • Newsletter
  • Customer Stories

Contact Us

USA:

phone-office +1 800 547 3000

See our Worldwide Directory

  • Contact Us
  • Support Center
  • Give us Feedback
©2021 Siemens Digital Industries Software. All Rights Reserved.
Terms of Use Privacy Cookie Policy