by Darron May, Product Marketing Manager, Verification Mgmt, Mentor Graphics and Abigail Moorhouse, R & D Manager, Mentor Graphics
If you can't measure something, you can't improve it. For years, verification engineers have used "coverage" as a way to measure completeness of the verification effort. Of course, there are many types of coverage, from different types of code coverage to functional coverage, as well as many tools, both dynamic and static, that provide coverage information. Simply put, coverage is a way to count interesting things that happen during verification and the measure of coverage is being able to correlate those things that happened back to a list of things you wanted to happen (also called a verification plan). Meaningful analysis requires a standard method of storing coverage data from multiple tools and languages, which is what Accellera's Unified Coverage Interoperability Standard (UCIS) finally delivers.
Last month, after five and a half years of hard work, the UCIS technical subcommittee approved draft version 1.0 of the standard, which the Accellera board will likely approve at DAC 2012, thus officially providing the industry with its first standard method of storing, analyzing and reporting coverage across multiple tools and languages. UCIS is an unequivocal accomplishment and a massive step toward producing a unified way of handling coverage. What did the process involve? And what should be expected from the standard? For answers, it's perhaps useful to look back at work done at Mentor Graphics to build a unified coverage database for our tools, work that took place long before the UCIS process started and that in fact has seeded Accellera's march toward better coverage interoperability.
HISTORY OF THE UCIS
Accellera's UCIS subcommittee was formed in November 2006 with two goals: identifying interoperability opportunities and defining coverage models and an interoperability standard for data exchange to encourage EDA innovation in the coverage space. The original group was made up of over 180 people from 18 member companies, including EDA companies and their users. There were many barriers to overcome, notably including aligning user and vendor requirements, and also dealing with nonstandard coverage models built with proprietary implementations of code coverage techniques and mixed languages. Even a coverage metric like SystemVerilog functional coverage, which is defined in a standard, differs slightly in implementation between vendors in identifying coverage in parameterizable classes. If coverage was ever to be unified between vendors, then every implementation would have to follow the same coverage representation.
Accordingly, during the first 18 months of the UCIS process, the subcommittee defined both a high level design of what a unified coverage database would need to look like and coverage terms. Next came a call for technology donations. Given the alignment between what the standard required and the capabilities of Mentor Graphics' Unified Coverage Database (UCDB) technology, we decided to donate this technology to the industry. A year later, after a separate donation by Synopsys of their own Unified Coverage API (UCAPI) technology and the addition of UCDB extensions from Cadence to support their data models, Mentor Graphics' UCDB was voted the basis of the UCIS standard. The last two years have been spent working through complex areas of the standard to ensure alignment with the initial user requirements and delivery of a standard that could be supported by all vendors.
HISTORY OF THE UCDB
Mentor Graphics' UCDB was architected in 2005 to unify coverage collection within all the company's verification tools. It was first released within Questa and ModelSim in early 2006 as a way of natively storing, analyzing and reporting on functional coverage, code coverage and assertions. The database has an open C Application Programming Interface (API) that has been used to develop all the UCDB utilities for Questa and ModelSim such as ranking, merging, analysis and reporting. The database not only ensured the definition of standard coverage models for all metrics generated by Mentor Graphics tools but also provided support of third party and user coverage, along with the ability to extend to coverage yet to be defined by any tools. This openness has allowed users to bring coverage data from other languages and vendors tools into the UCDB and thus use the database to completely unify their coverage storage and analysis within their flows. After its initial release, other Mentor Graphics product groups added their support for the UCDB, including those focusing on formal verification, analog design and verification, and emulation.
The database was architected to support the management of verification and so, quite logically, became the basis of the Questa Verification Management (VM) tools. These tools allow data management, and also process and tool automation to improve verification productivity, specifically in the areas of capacity, throughput and turnaround time.
A complete verification management solution requires a database to store unified coverage; low level utilities to manage this data, including merging, ranking, analysis and reporting; and higher level applications, such as testplan tracking, trend analysis, results analysis and run management automation. While Mentor Graphics has been actively involved with the UCIS over the last few years, the company has been simultaneously building applications based around the UCDB.
From the users' point of view, the database has three primary sections. One is the coverage data collection area. The second is for recording of test-specific information, such as test name, tool settings, CPU time, username and so on, and is extendable to store any user defined information. This section also includes the history of how the tests were generated, combined and/or merged together. The third section, used for testplan tracking, allows the storage of testplan items that can be linked to the coverage model and/or the test cases themselves.
Designs and testbenches are hierarchically organized. Design units (Verilog modules or VHDL entity/architectures) can be hierarchical, though they are not always. Test plans can be hierarchical. Even coverage data (of which the SystemVerilog covergroup is the best example) can be hierarchically organized.
The UCDB handles hierarchical structures using scopes (also referred to as hierarchical nodes), which store hierarchical structures (i.e., elements of a database that can have children). Coverage and assertion data are stored as counters, which indicate how many times something happens in the design.
The basic design of coverage hierarchy is shown below. Note that the counters count how many times a sequence completed, how many times a bin incremented or how many times a statement executed. In UCDB terminology, these types of counters and some associated data are called coveritems. These counters are database leaf nodes, which cannot have children.
One of the major benefits of the system is that it allows multiple tests to be merged together into a single database while still keeping an association between which testcase incremented which bin. This optimization reduces the amount of stored data but preserves the important test association information needed for analysis. It also allows a vendor to add great value in the way the database is organized and optimized. A prime example is Questa, which, after more than six years of real world optimizations, today delivers capacity and performance in a platform-independent database.
MENTOR GRAPHICS' SUPPORT FOR UCIS
Mentor Graphics today offers a prototype UCIS API library capable of running the examples in the UCIS draft standard and a lot more. We also have a clear migration path from the original proprietary UCDB to the UCIS API that will allow users to continue to access existing UCDB databases.
Indeed, we are already ahead of the UCIS with feature extensions like testplan management that will be donated to the standard in the next round (or in some cases, have already been donated, but were not included). These extensions are of course available now to Mentor Graphics' users. Both the UCIS and UCDB APIs provide views into and control of otherwise opaque databases. The same database can support both API views, although the APIs cannot be mixed within the same application.
Due to the source of the standard it is no surprise that Mentor Graphics also models coverage in the database using many of the data models published in the standard, sometimes with minor differences. Migration to complete data compliance is a longer process and will probably occur on an as-needed basis. The database does not inherently constrain the data that may be loaded into it so all models are technically UCIS-compliant. Universal Object Recognition (UOR) offers an additional guarantee that a specified naming convention has been applied though is not currently supported for all the coverage models.
The release of UCIS 1.0 is the first important step towards unifying the coverage generated by multiple tools and vendors. Mentor Graphics has been trailblazing in this area for many years with its development of UCDB and applications built around the database that have become the basis of the standard. We will continue to support the UCIS efforts and ensure that other UCDB optimizations and features are put forward when the standard goes through its revisions. Questa users today have complete access to the UCIS API by the use of a prototype UCIS API library. And of course since Mentor Graphics complete verification management tool set is based around UCDB (and therefore UCIS), our users are already benefiting from infrastructure we built around the database even without using the UCIS API.
Back to Top