Functional Verification Study Survey Methodology Evolution

Beginning in 2024, the Siemens EDA and Wilson Research Group Functional Verification Study survey shifted from industry-list recruitment to a qualified sample-provider model, creating a break in direct comparability with studies conducted from 2007 through 2022. The wording of recurring questions remained consistent across the series, while additional questions were added as industry technologies and practices evolved. While exact values across the methodology transition should be interpreted cautiously, directional patterns within both methodological periods remain useful. The newer approach appears to have broadened participation, including greater representation from professionals with system-level responsibilities. Certain questions in the 2024 study experienced elevated item-level nonresponse, but the 2026 study demonstrated substantially stronger response completeness, with most questions having nonresponse rates between 0% and 5% and a maximum rate of approximately 10%.

Overview

The Siemens EDA and Wilson Research Group Functional Verification Study has tracked developments in semiconductor design and verification since 2007. Over that period, both the industry and the population of professionals involved in complex designs have changed significantly.

Beginning with the 2024 study, the survey adopted a new participant-recruitment methodology. This paper explains that transition, discusses its implications for interpreting historical results, and describes the item-level nonresponse issue identified in portions of the 2024 study.

The purpose of this paper is not to invalidate prior findings or diminish the value of the survey series. Rather, it is to clarify how the results should be interpreted and to distinguish between precise numerical comparisons and broader directional evidence.

Methodology Transition

From 2007 through 2022, participants were recruited from a collection of relevant industry lists, including Verification Academy membership. This approach provided continuity across the historical survey series and reached a well-established community of design and verification professionals.

Beginning in 2024, participants were recruited through a sample provider. Prospective respondents were required to satisfy qualifying criteria designed to confirm that they had relevant industry experience and responsibilities. The 2026 study continued to use this general recruitment and qualification approach. The wording of recurring questions remained unchanged across the study series, while additional questions were introduced over time to address emerging technologies, methods, and industry challenges. This continuity helps preserve consistency in what the recurring questions measure, although it does not remove the potential effects of the 2024 change in recruitment source and participant composition.

Because the source and composition of the participant pool changed, the 2024 study represents a methodological transition. Results collected before and after that transition should not be treated as perfectly equivalent measurements of the same population.

The survey series is best understood as two methodological periods:

Comparisons within each period are more directly aligned than comparisons across the 2022–2024 transition.