seed concept : Ex: I ran single test 9 times passing out of 10 times and one time is failing. Please help me, How to overcome from this issue
In reply to Subbi Reddy:
Are you running the same test with the same seed? or do you change the seed from test to test.
Yes,same seed using for 10 times
In reply to Subbi Reddy:
I can’t believe it, because the randomization is repeatable for the same seed. If you do not change the seed the randomization results always in the same values. What you are saying might point to some unstability in your design (metastable register values etc.).
In interview asked the this scenario,I quit the question due to lack of knowledge in this scenario. please help me if you know how to debug this scenario.
In reply to Subbi Reddy:
There is nothing to debug. This cannot happenbecause the test run is always the same.
I don’t agree.
In a sufficiently complex setup, there can always be uncontrolled variables unknown to the test bench writer which can cause these sorts of failures. This happens all of the time in real world scenarios. The random seed input is only one item that controls the behavior of a random test. A few other items that could cause different behavior:
- Running on a grid system that submits jobs to different machines with slightly different setups
- Unnoticed change in RTL or test bench code between runs
- An test bench input file (memory content file, for example) changes between runs
- Bug in the simulator or the base test code package
- Bug in the scripts being used to compile and run simulation jobs
This is a pretty good interview question, frankly. Aside from debugging gate-level simulations I would have to say that debugging intermittent failures due to random stability issues has always been one of my least favorite verification activities!
May i know any solution: Any alternate to overcome from this problem and How to debug like these scenarios??
In reply to Subbi Reddy:
I agree with Jon, this is a very good interview questions. The interviewer was not looking for a solution, instead they’re looking for the process you would follow to find a solution. That would include isolating the test and eliminating other possible causes. Did the tests all run on the same machine with the same version of software? Did they run in parallel or sequentially? At what point did the test fail and how did it fail? Etc.
In reply to Subbi Reddy:
Other reason is race scenario in the design.
Best way to debug is log comparing, especially the timing part.
You have 9 pass and 1 fail.
So compare the 1 fail log with 1 pass log, see the messages are the same or not.
And if messages are same, is the time the same.
Sometimes because of the race scenario you got the timing little changed, which makes the test-case to fail