I have two questions related to coverage of Assertion Based Verification:
What is the recommended way to report and observe that all coverage is at 100% while no assertions failed and no tests failed in the scoreboard? I would prefer to see a % reported for each module in our SOC, with all three of these items combined.
If I am using assertions to verify some of my output signals, such as asserting that an output is 1 only when enabled, and when a timer expires, it seems to me that I also need a cover statement to ensure the inputs have been stimulated in all conditions (enable is tested both low and high, and the timer signal has been expired & non-expired). Is there some documentation with examples of best practices for assertions and the corresponding cover statements?
I have two questions related to coverage of Assertion Based Verification:
What is the recommended way to report and observe that all coverage is at 100% while no assertions failed and no tests failed in the scoreboard? I would prefer to see a % reported for each module in our SOC, with all three of these items combined.
I see three ways to get coverage information:
While in simulation, the tool (most, if not all simulators) provide those statistics.
During the simulation run command, most tools provide a coverage report command that you can fire with various options. From Questa’s documentation, “The coverage report command produces textual output of coverage statistics or exclusions (e.g., assertions and cover groups). By default, the command prints results to the Transcript window, and returns an empty string. You can use the -file argument to save the output to a file”. See your simulator documentation for the specific command and options; this forum tries to be vendor independent, thus I won’t discuss the specifics.
If I am using assertions to verify some of my output signals, such as asserting that an output is 1 only when enabled, and when a timer expires, it seems to me that I also need a cover statement to ensure the inputs have been stimulated in all conditions (enable is tested both low and high, and the timer signal has been expired & non-expired). Is there some documentation with examples of best practices for assertions and the corresponding cover statements?
Asserting that an output is 1 only when enabled, and when a timer expires
[Ben] That’s an assertion
a cover statement to ensure the inputs have been stimulated in all conditions (enable is tested both low and high, and the timer signal has been expired & non-expired.
[Ben] That looks like cover sequence of several types of sequences.
Thanks for the response. Good info on API’s. I was hoping not to have to customize my reporting, since it seems like a common task that Questa should roll up all coverage and assertion grading into a single number per module, so I can scan a report and see the complete status of each module in one column, including coverage and test failures. I think I’ll take my first question over to a Modelsim/Questa support site.
My second question again seems like a common problem which I assume someone should have written a page of documentation regarding pairing assertions and cover statements, where the cover statements ensure the inputs to the logic are stimulated for all cases, and the assertion ensures that the outputs are correct for the given input conditions. Am I using assertions in a non-standard way? Or perhaps this is a basic occurrence which requires no detailed discussion. I was hoping to glean some hints on a “formula” for ensuring that I don’t miss something in my cover statement, based on a given assertion statement.
In reply to ben@SystemVerilog.us:
My second question again seems like a common problem which I assume someone should have written a page of documentation regarding pairing assertions and cover statements, where the cover statements ensure the inputs to the logic are stimulated for all cases, and the assertion ensures that the outputs are correct for the given input conditions. Am I using assertions in a non-standard way? Or perhaps this is a basic occurrence which requires no detailed discussion. I was hoping to glean some hints on a “formula” for ensuring that I don’t miss something in my cover statement, based on a given assertion statement.
That might be your lucky day :)
It turns out that I did address this topic in my SVA Handbook 4th Edition; link provides pages from that book. SystemVerilog.us/svabk4_coverage.pdf
Keep in mind that tools provide some coverage statistics on assertions (oassed, failed), but the cover property provides a more (see the pdf file).
Ben Cohen http://www.systemverilog.us/ben@systemverilog.us
Thanks Ben, indeed your paper does address my concern that both assertions and coverage need to be evaluated before considering a design verification “covered”. Your API example might be the best way to coordinate both of these reports, as you point out it is then simulator-independent. I’m still hoping that I find out that Questa has a way to combine the two results without using an API…
Ben,
A suggestion for your next edition of your book-- add a few examples of an assertion and a related cover statement, so that the reader gets the idea of how the two are related. Then someone might more easily see the benefit of using your API to combine the two under a combined reporting number.
Other readers using Questasim (who are as naive as I am about coverage reporting):
I did locate Questasim User Manual documentation regarding Aggregate coverage reporting (cover + assertion reporting), in the Coverage Aggregation link found here: https://supportnet.mentor.com/docs/201604008/docs/htmldocs/mgchelp.htm#context=questa_sim_user&id=960&tag=id52585b19-cb2f-417e-9962-73172199ae8b
The “coverage binning and calculation” page linked within the “calculation and total coverage” from the above page states that assertions are aggregated as they do not fail and occur non-vacuously.
Make sure to enable Structure>Code_Coverage>Enable_Recursive_Coverage_Sums for hierarchical accumulation of coverage.
The columns can be enabled in the Structure pane by right-clicking in a column heading, and select “Change Column Visibility”.
I found these columns useful: Instance, Visibility, Top Category, Total Coverage, Cover%, Assertion%, Covers hit, Covers Missed, Cover graph, Assertions hit, Assertions missed, Assertion graph.
Your structure pane settings can be saved by right-clicking in a column heading, select “Export Column Configurations.”
I also checked that a merged UCDB can indeed be displayed with the expanded-columns Structure pane. Very nice.
In reply to ted.firlit:
Since you’re getting a deep assertion coverage experience on a complex design, I strongly suggest that you write a paper on this topic. Yo can show this paper at the Verification Horizons, or at a DVCon, or at other seminars.
Do consider it!
Ben