Verification Academy

Search form

My Account Menu

  • Register
  • Log In
  • Topics
  • Courses
  • Forums
  • Patterns Library
  • Cookbooks
  • Events
  • More
  • All Topics
    The Verification Academy offers users multiple entry points to find the information they need. One of these entry points is through Topic collections. These topics are industry standards that all design and verification engineers should recognize. While we continue to add new topics, users are encourage to further refine collection information to meet their specific interests.
    • Languages & Standards

      • Portable Test and Stimulus
      • Functional Safety
      • Design & Verification Languages
    • Methodologies

      • UVM - Universal Verification Methodology
      • UVM Framework
      • UVM Connect
      • FPGA Verification
      • Coverage
    • Techniques & Tools

      • Verification IP
      • Simulation-Based Techniques
      • Planning, Measurement, and Analysis
      • Formal-Based Techniques
      • Debug
      • Clock-Domain Crossing
      • Acceleration
  • All Courses
    The Verification Academy is organized into a collection of free online courses, focusing on various key aspects of advanced functional verification. Each course consists of multiple sessions—allowing the participant to pick and choose specific topics of interest, as well as revisit any specific topics for future reference. After completing a specific course, the participant should be armed with enough knowledge to then understand the necessary steps required for maturing their own organization’s skills and infrastructure on the specific topic of interest. The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization’s processes so that you can then reap the benefits that advanced functional verification offers.
    • Universal Verification Methodology (UVM)

      • Advanced UVM
      • Basic UVM
      • Introduction to UVM
      • UVM Connect
      • UVM Debug
      • UVMF - One Bite at a Time
    • Featured Courses

      • Introduction to ISO 26262
      • Introduction to DO-254
      • Clock-Domain Crossing Verification
      • Portable Stimulus Basics
      • Power Aware CDC Verification
      • Power Aware Verification
      • SystemVerilog OOP for UVM Verification
    • Additional Courses

      • Assertion-Based Verification
      • An Introduction to Unit Testing with SVUnit
      • Evolving FPGA Verification Capabilities
      • Metrics in SoC Verification
      • SystemVerilog Testbench Acceleration
      • Testbench Co-Emulation: SystemC & TLM-2.0
      • Verification Planning and Management
      • VHDL-2008 Why It Matters
    • Formal-Based Techniques

      • Formal Assertion-Based Verification
      • Formal-Based Technology: Automatic Formal Solutions
      • Formal Coverage
      • Getting Started with Formal-Based Technology
      • Handling Inconclusive Assertions in Formal Verification
      • Sequential Logic Equivalence Checking
    • Analog/Mixed Signal

      • AMS Design Configuration Schemes
      • Improve AMS Verification Performance
      • Improve AMS Verification Quality
  • All Forum Topics
    The Verification Community is eager to answer your UVM, SystemVerilog and Coverage related questions. We encourage you to take an active role in the Forums by answering and commenting to any questions that you are able to.
    • UVM Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • UVM Forum
    • SystemVerilog Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • SystemVerilog Forum
    • Coverage Forum

      • Active Questions
      • Solutions
      • Replies
      • No Replies
      • Search
      • Coverage Forum
    • Additional Forums

      • Announcements
      • Downloads
      • OVM Forum
  • Patterns Library
    The Verification Academy Patterns Library contains a collection of solutions to many of today's verification problems. The patterns contained in the library span across the entire domain of verification (i.e., from specification to methodology to implementation—and across multiple verification engines such as formal, simulation, and emulation).
    • Implementation Patterns

      • Environment Patterns
      • Stimulus Patterns
      • Analysis Patterns
      • All Implementation Patterns
    • Specification Patterns

      • Occurrence Property Patterns
      • Order Property Patterns
      • All Specification Patterns
    • Pattern Resources

      • Start Here - Patterns Library Overview
      • Whitepaper - Taking Reuse to the Next Level
      • Verification Horizons - The Verification Academy Patterns Library
      • Contribute a Pattern to the Library
  • All Cookbooks
    Find all the methodology you need in this comprehensive and vast collection. The UVM and Coverage Cookbooks contain dozens of informative, executable articles covering all aspects of UVM and Coverage.
    • UVM Cookbook

      • UVM Basics
      • Testbench Architecture
      • DUT-Testbench Connections
      • Configuring a Test Environment
      • Analysis Components & Techniques
      • End Of Test Mechanisms
      • Sequences
      • The UVM Messaging System
      • Other Stimulus Techniques
      • Register Abstraction Layer
      • Testbench Acceleration through Co-Emulation
      • Debug of SV and UVM
      • UVM Connect - SV-SystemC interoperability
      • UVM Versions and Compatibility
      • UVM Cookbook
    • Coding Guidelines & Deployment

      • Code Examples
      • UVM Verification Component
      • Package/Organization
      • Questa/Compiling UVM
      • SystemVerilog Guidelines
      • SystemVerilog Performance Guidelines
      • UVM Guidelines
      • UVM Performance Guidelines
    • Coverage Cookbook

      • Introduction
      • What is Coverage?
      • Kinds of Coverage
      • Specification to Testplan
      • Testplan to Functional Coverage
      • Bus Protocol Coverage
      • Block Level Coverage
      • Datapath Coverage
      • SoC Coverage Example
      • Requirements Writing Guidelines
      • Coverage Cookbook
  • All Events
    No one argues that the challenges of verification are growing exponentially. What is needed to meet these challenges are tools, methodologies and processes that can help you transform your verification environment. These recorded seminars from Verification Academy trainers and users provide examples for adoption of new technologies and how to evolve your verification process.
    • Upcoming & Featured Events

      • Low Power Verification - 4/29
      • Fault Campaign for Mixed-Signal - 5/4
      • User2User - 5/26
      • Webinar Calendar
    • On-Demand Webinars

      • CDC+RDC Analysis
      • Basic Abstraction Techniques
      • Safety Analysis Techniques
      • QVIP Workflow and Debug for PCIe
      • Writing a Proxy-driven Testbench
      • Achieving High Defect Coverage
      • Visualizer Features
      • All On-Demand Webinars
    • Recording Archive

      • Siemens EDA 2021 Functional Verification Webinar Series
      • Improving Your SystemVerilog & UVM Skills
      • Should I Kill My Formal Run?
      • Visualizer Debug Environment
      • Industry Data & Surveys
      • All Recordings
    • Conferences

      • DVCon 2021
      • DVCon 2020
      • DAC 2019
      • All Conferences
    • Mentor Learning Center

      • SystemVerilog Fundamentals
      • SystemVerilog UVM
      • View all Learning Paths
  • About Verification Academy
    The Verification Academy will provide you with a unique opportunity to develop an understanding of how to mature your organization's processes so that you can then reap the benefits that advanced functional verification offers.
    • Blog & News

      • Verification Horizons Blog
      • Academy News
      • Academy Newsletter
      • Technical Resources
    • Verification Horizons Publication

      • Verification Horizons - March 2021
      • Verification Horizons - November 2020
      • Verification Horizons - July 2020
      • Issue Archive
    • About Us

      • Verification Academy Overview
      • Subject Matter Experts
      • Contact Us
    • Training

      • Questa Basic
      • Questa Advanced
      • Mastering Questa
  • Home
  • Verification Horizons
  • November 2018
  • FPGA Verification Challenges and Opportunities

FPGA Verification Challenges and Opportunities

Verification Horizons - Tom Fitzpatrick, Editor

FPGA Verification Challenges and Opportunities Harry Foster - Mentor, A Siemens Business

INTRODUCTION

There have been multiple studies on IC/ASIC functional verification trends published over the years.[1][2][3][4] However, there are no published studies specifically focused on Field-Programmable Gate Array (FPGA) verification trends. To address this dearth of information, this article highlights a few key FPGA findings from the 2018 Wilson Research Group Functional Verification Study. The findings from this study provide invaluable insight into the state of today’s FPGA market in terms of functional verification.

The Global Semiconductor Market

The global semiconductor market was valued at $444.70 billion in 2017, of which, $4.7 billion is accounted for by FPGAs.[5][6] The FPGA market is expected to reach a value of $8.8 billion by 2027, growing at a compounded annual growth rate (CAGR) of 6.4% during this forecast period. The growth in this market is being driven by new and expanding end-user applications related to automotive, IoT, telecommunication, industrial, mil/aero, consumer, and emerging AI applications within the data center requiring acceleration.

Historically, FPGAs have offered two primary advantages over ASICs. First, due to their low NRE[7], FPGAs are generally more cost effective than IC/ASICs for low-volume production. Second, FPGAs’ rapid prototyping capabilities and flexibility can reduce the development schedule since a majority of the verification and validation cycles have traditionally been performed in the lab. More recently, FPGAs offer advantages related to performance for certain accelerated applications by exploiting hardware parallelism (e.g., AI Neural Networks).

Growing Design Complexity

The IC/ASIC market in the mid- to late-2000 timeframe underwent growing pains to address increased verification complexity. Similarly, we find today’s FPGA market is being forced to address growing verification complexity. With the increased capacity and capability of today’s complex FPGAs, and the emergence of high-performance SoC programmable FPGAs (e.g., Xilinx® Zynq®, Intel® Arria®, Cyclone®, and Stratix®, along with Microsemi® SmartFusion®), traditional lab-based approaches to FPGA verification and validation are becoming less effective. In this article, we quantify the ineffectiveness of today’s FPGA verification processes in terms of non-trivial bug escapes into production.

FPGA VERIFICATION EFFECTIVENESS

IC/ASIC projects have often used the metric “number of required spins before production” as a benchmark to assess a project’s verification effectiveness. Historically, about 30% of IC/ASIC projects are able to achieve first silicon success, and most successful designs are productized on the second silicon spin. Unfortunately, FPGA projects have no equivalent metric. As an alternative to IC/ASIC spins, our study asked the FPGA participants “how many non-trivial bugs escaped into production?” The results shown in Fig. 1 are somewhat disturbing. In 2018, only 16% of all FPGA projects were able to achieve no bug escapes into production, which is worse than IC/ASIC in terms of first silicon success, and for some market segments, the cost of field repair can be significant.

Figure 1 - Non-trivial FPGA bug escapes into production


For example, in the mil-aero market, once a cover has been removed on a system to upgrade the FPGA, the entire system needs to be revalidated.

FPGA VERIFICATION EFFORT

In this section, we discuss trends in terms of FPGA project time and resources.

Percentage of Project Time Spent in Verification

Fig. 2 shows the percentage of total FPGA project time spent in verification. You can see two extremes in this graph. In general, projects that spend very little time in verification are typically working on designs with a good deal of existing pre-verified design IP, which is integrated to create a new product. On the other extreme, projects that spend a significant amount of time in verification often have a high percentage of newly developed design IP that must be verified.

Figure 2 - Percentage of FPGA project time spent in verification


Overall, we found an increase in average percentage of FPGA project time spent in verification during the period 2014 through 2018. This is an indication of growing design and verification complexity.

Mean Peak Number of Engineers

Perhaps one of the biggest challenges today is to control cost and engineering headcount, which means identifying FPGA design and verification solutions that increase productivity. To illustrate the need for productivity improvement, we discuss the trend in terms of increasing engineering headcount. Fig. 3 shows the mean peak number of FPGA engineers working on a project.

While, on average, the demand for design engineers is growing at about a 4% CAGR (which is similar growth for IC/ASIC), the demand for verification engineers is growing at about a 10% CAGR. It is worth noting that during the period 2007 through 2014, the IC/ASIC market went through similar growth demands related to verification engineers to address growing verification complexity.[3]

Figure 3 - Mean peak number of FPGA engineers on project


FPGA VERIFICATION ADOPTION TRENDS

To address growing verification complexity, we find that many FPGA projects have been forced to mature their verification processes. In this section, we present FPGA trends related to the adoption of various verification techniques, which are fairly standard practice today on most IC/ASIC projects.

The adoption trends for formal property checking (e.g., model checking) and automatic formal applications are shown in Fig. 4. We found that the adoption of formal property checking on FPGA projects is growing at an impressive 21% CAGR, and the adoption of automatic formal applications is growing at a 29% CAGR. Historically, the formal property checking process has required specialized skills and expertise. However, the recent emergence of automatic formal applications provides narrowly focused solutions and does not require specialized skills for adoption. In general, formal solutions (i.e., formal property checking combined with automatic formal applications) are one of the fastest growing segments in functional verification.

Figure 4 - FPGA project formal technology adoption trends


Fig. 5 shows the FPGA project adoption trends for various simulation-based techniques from 2012 through 2018, which include code coverage, functional coverage, assertions, and constrained-random simulation.

Figure 5 - FPGA project simulation technique trends


CONCLUSION AND DISCUSSION

In this article, we presented FPGA design and verification trends based on a recent, large industry study. FPGAs have grown in complexity equal to many of today’s IC/ASIC designs. We quantified the impact of this growing complexity in terms of verification effectiveness and effort.

Perhaps the most disturbing finding from this year’s study relates to the number of FPGA projects with non-trivial bug escapes into production. We did find an interesting correlation between the improvement of reduced functional flaws contributing to non-trivial bug escapes, as shown in Fig. 1, and the maturing of FPGA projects’ functional verification processes. The data suggest that projects that are more mature in their functional verification processes will likely experience fewer bug escapes. To test this claim, we partitioned the study participants into two groups: FPGA projects with no bug escapes and FPGA projects that experienced a bug escape. We then examined the percentage adoption of various verification techniques and the results are shown in Fig. 6. These findings are statistically significant in that the group with no bug escapes tended to have higher adoption of various verification techniques, which suggest they are more mature in their verification process. However, what we are unable to measure from our study is how effective a project was in adopting any of these processes. For example, a project that experienced a bug escape could claim that they have adopted functional coverage, yet the fidelity of their functional coverage model might be poor due to their inexperience. From our study data, we are unable to assess successful or effective adoption for any particular verification technique.

Figure 6 - FPGA simulation technique adoption versus non-trival bug escapes


REFERENCES

  • [1] R. Collett, “2002 IC/ASIC functional verification study,” Industry Report from Collett International Research, Inc. 2003.
  • [2] R. Collett, “2004 IC/ASIC functional verification study,” Industry Report from Collett International Research, Inc. 2005.
  • [3] H. Foster, Trends in functional verification: a 2014 industry study, Proceedings of the 52nd Annual Design Automation Conference, p.1-6, June 07-11, 2015, San Francisco, California.
  • [4] H. Foster, Trends in functional verification: a 2016 industry study, DVCon 2017, San Jose, California.
  • [5] IC Insights, The Mid-Year Update to the McClain Report, 2018.
  • [6] International Business Strategies, Semiconductor Market Analysis, 2017 Review, 2018 Projections, February 14, 2018.
  • [7] S. Trimberger, Three ages of FPGAs: a retrospective on the first thirty years of FPGA Technology, Proceedings of the IEEE, Vol 103, Issue 3, March 2015.

Back to Top

Table of Contents

Verification Horizons Articles:

  • Those Who Left Us with Much, also Left Much Too Soon

  • FPGA Verification Challenges and Opportunities

  • Building a Better Virtual Sequence with Portable Stimulus

  • A New Approach to Low-Power Verification: Power Aware Apps

  • Simplifying Mixed-Signal Verification

Siemens Digital Industries Software

Siemens Digital Industries Software

##TodayMeetsTomorrow

Solutions

  • Cloud
  • Mendix
  • Siemens EDA
  • MindSphere
  • Siemens PLM
  • View all portfolio

Explore

  • Digital Journeys
  • Community
  • Blog
  • Online Store

Siemens

  • About Us
  • Careers
  • Events
  • News and Press
  • Newsletter
  • Customer Stories

Contact Us

USA:

phone-office +1 800 547 3000

See our Worldwide Directory

  • Contact Us
  • Support Center
  • Give us Feedback
©2021 Siemens Digital Industries Software. All Rights Reserved.
Terms of Use Privacy Cookie Policy