2018 Wilson Research Group: Functional Verification Study

2018 Wilson Research Group
Functional Verification Study

Harry D. Foster
Chief Scientist Verification
IC Verification Solutions (ICVS)
Aug 2018
A Tale of Two Technologies
ASIC & FPGA Functional Verification Trends
It was the best of times, it was the worst of times...

A Tale of Two Cities — Charles Dickens
© Mentor Graphics Corporation

Worldwide IC Market Growth
50%
40%
35% 33%
30% 28%
25%
20%
16%
IC Market Change
19% 16%
10% 8% 9%
4% 7% 4%
5%
2% 1% 3%
0%
-2%
-10% -9% -6% -4%
-10%
-20%
-30%
-33%
-40%
98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18F 19F
Year
Source: IC Insight, McClean Report 2018 (June Update), A Complete Analysis and Forecast of the Integrated Circuit Industry
Overall World Wide Semiconductor Market
$600
$500
$509.10
Semiconductor Market ($B)
$400
$444.70
$300
$364.00
$200
$100
$0
2016 2017 2018F
Source: IC Insight, McClean Report 2018 (June Update), A Complete Analysis and Forecast of the Integrated Circuit Industry
Global FPGA Market
$10.00
$9.00
$8.00
$8.80
FPGA Market ($B)
$7.00
$6.00
$5.00
$4.00
$4.70
$3.00
$2.00
$1.00
$0.00
2017 2027F
Source: Statistics MRC 2018

5 HF, 2018 Wilson Research Group Functional Verification Study, Aug 2018. FOR LIMITED RELEASE
FPGA Applications
Data Processing Automotive Mil/Aero
Consumer Telecommunications Industrial

ASIC: Number of Required Spins Before Production
50% 2012
2014
2016
40%
2018
Design Projects
30%
20%
10%
0%
1 (FIRST SILICON 2 3 4 5 6 7 SPINS or MORE
SUCCESS)
Number of Required ASIC Spins Before Production
Source: Wilson Research Group and Mentor, A Siemens Business, 2018 Functional Verification Study
FPGA: Non-trivial Bug Escapes into Production
35% 84% of FPGA design projects 2016
have non-trivial bugs escape 2018

30% into production
25%
Design Projects
20%
15%
10%
5%
0%
0 1 2 3 4 5 6 or More
Non-trivial FPGA Bug Escapes into Production
Study Background
 Worldwide study
— North and South America, Europe, Mideast, Africa, China, India, Pac Rim, Australia, Japan
 Sample frame consisted of 1205 participants

— 31% smaller than our 2016 study (1738)
— 36% smaller than our 2014 study (1886)
— 89% larger than our 2012 study
— 5.4x larger than the 2004 Ron Collett International study
 Confidence interval 95%

— ±4% Margin of Error
9 H Foster, WRG Functional Verification Study, July 2016
Section A.
ABOUT PARTICIPANT’S COMPANY

2018 Study Participation by Market Segment
PC/Workstation/Server/Mainframe
Peripherals (printers, monitors, storage devices, UPSs, etc.)
Office Equipment (copiers, etc.)
Wireless: cellular handsets, cordless phones, wireless LANs, radios, pagers,…
Wireless: base stations, satellite, TV transmission, radar
Networking: backbone routers, switches, other WAN products
FPGA
Networking: Optical products
ASIC/IC
Networking: hubs, NIC, LAN routers, bridges, other LAN products
Networking: fax, modems, video-conferencing, set-top boxes, internet phones
Consumer Audio, Video (TVs, DVDs, cameras, etc.), Games
Automotive
Aerospace/Military
Medical equipment
Household Appliances
Industrial Controls
Other
0% 5% 10% 15% 20%
2016 vs 2018 ASIC Study Participation by Market Segment
2016
2018
Automotive
Aerospace/Military
Medical equipment
Industrial Controls
Other
0% 5% 10% 15% 20%
2016 vs 2018 FPGA Study Participation by Market Segment
2016
2018
Automotive
Aerospace/Military
Medical equipment
Industrial Controls
Other
0% 5% 10% 15% 20% 25%
2018 Study Participation by Job Title
Hardware designer
Verification engineer
Emulation engineer
FPGA Prototyping Engineer
Post-Silicon Validation Engineer
Product engineer
Test engineer
Safety Engineer
System architect FPGA

ASIC/IC
Team leader
Software engineer
Manager
CAD support engineer
Other
0% 5% 10% 15% 20% 25% 30% 35%
2016 vs. 2018 ASIC Study Participation by Job Title
Hardware designer
Emulation engineer
Product engineer
Test engineer
Safety Engineer
System architect
Team leader
Software engineer
2016
Manager
2018
Other
0% 5% 10% 15% 20% 25% 30% 35% 40%
2016 vs. 2018 FPGA Study Participation by Job Title
Hardware designer
Emulation engineer
Product engineer
Test engineer
Safety Engineer
System architect
Team leader
Software engineer
2016
Manager
2018
Other
0% 5% 10% 15% 20% 25% 30% 35% 40%
2018 Functional Verification Study Participation by Region
North America
Sourth America
Europe
Mideast / Africa
China
India
Japan
PacRim / Austraila
0% 5% 10% 15% 20% 25% 30% 35% 40%
2018 Ratio of ASIC vs FPGA Projects by Region
North America
Sourth America
Europe
Mideast / Africa
ASIC
China FPGA
India
Japan
PacRim / Austraila
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Section B.
PARTICIPANT’S CURRENT DESIGN

Study Participation by Targeted Implementation
FPGA
High-Performance SoC FPGA
Gate array
Embedded array
FPGA 2016
Standard cell
FPGA 2018
ASIC 2016
Structured ASIC
ASIC 2018
Structured custom
Full custom
Other
0% 5% 10% 15% 20% 25% 30%

Design Projects
High-Performance SoC FPGA Adoption Percentage by Type
Xilinx Zynq
Altera Arria
Altera Cyclone
41% of FPGA projects are
Altera Stratix implementing with
programmable SoC FPGAs
Microsemi's
SmartFusion
Other Programmable
SoC FPGA
0% 10% 20% 30% 40% 50% 60%
Current High-Performance SoC FPGA Type Being Implemented by Project

ASIC Study Participation by Gate Count (Design Size)
2014: 18% < 500K Gates 2014
20% 2016: 21% < 500K Gates 2016
2018: 29% < 500K Gates 2018
15%
Study Participants
10%
5%
0%
Gates of Logic and Datapath Excluding Memories

ASIC Study Participation by Gate Count (Design Size)
2014
20% 2016
2018
15%
Study Participants
10%
5%
0%
Gates of Logic and Datapath Excluding Memories

ASIC: Study Participants by Design Size
40%
36%
< 1M
35%
33% 1M-40M
31%
30% >40M
Design Projects
25%
20%
15%
10%
5%
0%
Design Size
Study Participation by Geometry Feature Size
25%
2016
2018
20%
Study Participants
15%
10%
5%
0%
0.15µ or 0.13µ 0.09µ (90nm) 0.065µ 0.045µ 0.032µ 0.028µ 0.022µ 0.014µ 0.010µ 0.007µ (7nm)
larger (65nm) (45nm) (32nm) (28nm) (22nm) (14nm) (10nm) or smaller
Geometry Feature Size

ASIC: Number of Embedded Microprocessors
40%
66% of designs contain embedded processors 2016
35% 2018
48% of designs contain 2 or more processors
30%
ASIC Design Projects
25%
20%
15%
10%
5%
0%
0 1 2 3 4 5 6 7 8 or more
Number of Embedded Microprocessors
ASIC: Mean Number of Embedded Microprocessors by Design Size
5
< 1M
4 1M-40M
>40M
Mean Number of Embedded Processors
2
2
1 1
ASIC: Mean Number of Embedded Microprocessors by Design Size

FPGA: Number of Embedded Microprocessors
40%
64% of designs contain embedded processors 2016

35%
2018
30%
FPGA Design Projects
25%
20%
15%
10%
5%
0%
0 1 2 3 4 5 6 7 8 or more
ASIC vs. FPGA: Number of Embedded Microprocessors
40% • 66% of ASIC designs contain 1 or more embedded processors
• 48% of ASIC designs contain 2 or more embedded processors
35%
• 64% of FPGA designs contain 1 or embedded processors

30%
• 43% of FPGA designs contain 2 or more embedded processors
Design Projects
25%
20%
FPGA
ASIC
15%
10%
5%
0%
0 1 2 3 4 5 6 7 8 or more
ASIC: ON-CHIP "internal" Bus Adoption
None used in this design
Proprietary
Arm AMBA APB
Arm AMBA AHB
Arm AMBA AXI
Arm AMBA ACE
Arm AMBA CHI
IBM CoreConnect
Sonics
TileLink (RISC-V)
Arteris FlexNoC
Arteris (Ncore)
Other
0% 5% 10% 15% 20% 25% 30% 35%

FPGA: ON-CHIP "internal" Bus Adoption
None used in this design
Proprietary
Arm AMBA APB
Arm AMBA AHB
Arm AMBA AXI
Arm AMBA ACE
Arm AMBA CHI
IBM CoreConnect
Sonics
TileLink (RISC-V)
Arteris FlexNoC
Arteris (Ncore)
Other
0% 5% 10% 15% 20% 25% 30% 35%

ASIC: Number of Embedded DSP Cores
70%
2016
2018
60%
50%
40%
30%
20%
10%
0%
0 1 2 3 4 5 or more
Number of Embedded DSP Cores
ASIC: Mean Number of Number of Embedded DSP Cores by Design Size
1.5 < 1M
1.4 1M-40M
>40M
Mean Number of Embedded DSP Cores
1.0
0.7
0.5 0.4
0.0
ASIC: Mean Number of Embedded DSP Cores by Design Size

FPGA: Number of Embedded DSP Cores
70%
2016
2018
60%
50%
40%
30%
20%
10%
0%
0 1 2 3 4 5 or more
Number of Embedded DSP Cores
ASIC: Projects Working on Safety Critical Design
70%
2016
2018
59%
60%
54%
50%
46%
41%
40%
30%
20%
10%
0%
Safety Critical Design Not Safety Critical
ASIC: Adoption for Specific Functional Safety Standards
DO-254 - Avionics
ISO26262 - Automotive
IEC61508 - Industrial
IEC61511 - Process Industry
IEC61513 - Nuclear
IEC60601 - Medical
EN50129 - Railway
ISO25119 - Agriculture &…
MIL-STD-882 - Military
Other
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Safety Critical ASIC Design Projects

** Multiple answers possible
FPGA: Projects Working on Safety Critical Design
70%
2016
2018
60%
56%
53%
50% 47%
44%
40%
30%
20%
10%
0%
Safety Critical Design Not Safety Critical
FPGA: Adoption for Specific Functional Safety Standards
DO-254 - Avionics
IEC61513 - Nuclear
IEC60601 - Medical
EN50129 - Railway
Other
0% 5% 10% 15% 20% 25% 30% 35%
Safety Critical FPGA Design Projects

Adoption for Specific Functional Safety Standards
DO-254 - Avionics
IEC61513 - Nuclear
IEC60601 - Medical FPGA

ASIC
EN50129 - Railway
Other
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Safety Critical Design Projects

ASIC: Design Projects Implementing Security Features
70%
2016
2018
60%
60% 58%
50%
42%
40%
40%
30%
20%
10%
0%
Security Features No Security Features
FPGA: Design Projects Implementing Security Features
70%
2016
2018
60%
54%
52%
50% 48%
46%
40%
30%
20%
10%
0%
Security Features No Security Features
Section C.
PROJECT RESOURCES

ASIC: Mean Peak Number of Engineers on a Project
12.0 11.6
Design Engineers 11.0 11.0
Verification Engineers 10.5 10.5
10.1
10.0
8.5 8.4
8.1
8.0 7.8 7.6
Design Projects
6.0
4.8
4.0
2.0
0.0
2007 2010 2012 2014 2016 2018
Mean Peak Number of Engineers on ASIC Projects
ASIC: Mean Peak Number of Engineers By Design Size
18
< 1M
16 1M-40M 15.25
Mean Peak Number of Engineers on Project
14.83
>40M
14
12
10.36
10 9.07
5.78
6 5.31
0
Design Engineers Verification Engineers
2018 Mean Peak Number of ASIC Engineers by Design Size
FPGA: Mean Peak Number of Engineers on a Project
6.0 • Design Engineers growing at a 3.79% CAGR

• Verification Engineers growing at a 9.98% CAGR
Mean Peak Number of Engineers on Project
5.0
5.0
4.6
4.3 4.3
4.0
4.0
3.5 3.6
3.0 2.6
2.0
1.0
0.0
2012 2014 2016 2018
Design Engineers
Mean Peak Number of Engineers on FPGA Projects
Verification Engineers
ASIC: Mean % Time Design Engineer is Doing Design vs Verification
100%
80%
47% 46%
53%
Design Projects
60%
Doing Design
Doing Verification
40%
53% 54%
20%
47%
0%
2014 2016 2018
Mean Percentage Time ASIC Design Engineer is Doing Design vs Verification
FPGA: Mean % Time Design Engineer is Doing Design vs Verification
100%
80%
44%
53% 51%
Design Projects
60%
Doing Design
Doing Verification
40%
56%
47% 49%
20%
0%
2014 2016 2018
Mean Percentage of Time FPGA Design Engineer is Doing Design vs Verification
ASIC: Where Verification Engineers Spend Their Time
44%
Test Planning
3%
Testbench Development
13% Creating Test and Running Simulation

Debug
21%
Other
19%
FPGA: Where Verification Engineers Spend Their Time
42%
Test Planning
4%
Testbench Development
14% Creating Test and Running Simulation

Debug
20%
Other
19%
ASIC: Percentage of Project Time Spent in Verification
2012
2012: Average 55%
2014: Average 57% 2014
25%
2016: Average 54% 2016
2018: Average 53% 2018
20%
Design Projects
15%
10%
5%
0%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%
Percentage of ASIC Project Time Spent in Verification
FPGA: Percentage of Project Time Spent in Verification
2012: Average 44% 2012
2014: Average 46%
25% 2014
2016: Average 49%
2018: Average 50%
2016
20%
2018
Design Projects
15%
10%
5%
0%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%
Percentage of FPGA Project Time Spent in Verification
ASIC: How Long is Your Current Project Schedule
Less than a month Median: 7-9 Months
1 - 3 months
4 - 6 months
7 - 9 months
10 - 12 months
13 - 18 months
19 - 24 months
Greater than 24 months
0% 5% 10% 15% 20% 25% 30%
2018 ASIC Design Projects

FPGA: How Long is Your Current Project Schedule
Less than a month Median: 7-9 Months
1 - 3 months
4 - 6 months
7 - 9 months
10 - 12 months
13 - 18 months
19 - 24 months
0% 5% 10% 15% 20% 25% 30%
2018 FPGA Design Projects

ASIC vs FPGA: How Long is Your Current Project Schedule
Less than a month ASIC
FPGA
1 - 3 months
4 - 6 months
7 - 9 months
10 - 12 months
13 - 18 months
19 - 24 months
0% 5% 10% 15% 20% 25% 30%
Design Projects
Number of FPGAs Completed by Project Team per Year
40%
2014
35%
2016
2018
30%
25%
Design Projects
20%
15%
10%
5%
0%
1 2 3 4 5 6 or more
Number of FPGAs Completed by Project Team per Year
Section D.
SIMULATION CHARACTERISTICS

ASIC: Typical Regression Time
2014
25% 2016
2018
20%
Design Projects
15%
10%
5%
0%
<5 hrs 5-<9 hrs 9-<17 hrs 17-<24 hrs 1-<1.5 days 1.5-<2 days 2-<3 days 3-<4 days 4-7 days > 7 days
Typical ASIC Regression Time

ASIC: Typical Regression Time
50%
2014
45%
2016
40% 2018
35%
30%
Design Projects
25%
20%
15%
10%
5%
0%
<9 hrs 9-48 hrs > 48hrs
Typical ASIC Regression Time
FPGA: Typical Regression Time
45%
2014
40%
2016
35% 2018
30%
Design Projects
25%
20%
15%
10%
5%
0%
Typical FPGA Regression Time

FPGA: Typical Regression Time
60%
2014
2016
50%
2018
40%
Design Projects
30%
20%
10%
0%
<9 hrs 9-48 hrs > 48hrs
Typical FPGA Regression Time
ASIC: Longest Regression Time
20%
2016
2018
15%
Design Projects
10%
5%
0%
Longest ASIC Regression Time

ASIC: Longest Regression Time
2016
40%
2018
30%
Design Projects
20%
10%
0%
<9 hrs 9-48 hrs > 48hrs
Longest ASIC Regression Time
FPGA: Longest Regression Time
35%
2016
30%
2018
25%
Design Projects
20%
15%
10%
5%
0%
Longest FPGA Regression Time

FPGA: Longest Regression Time
60%
2016
50% 2018
40%
Design Projects
30%
20%
10%
0%
<9 hrs 9-48 hrs > 48hrs
Longest FPGA Regression Time
Section E.
LANGUAGES AND LIBRARIES

ASIC: Design Language Adoption Next Twelve Months
90%
2012
80% 2014
2016
70% 2018
Next Year
60%
Design Projects
50%
40%
30%
20%
10%
0%
VHDL Verilog SystemC SystemVerilog C/C++ OTHER Design
ASIC Design Language Adoption ** Multiple answers possible
FPGA: Design Language Adoption Next Twelve Months
80%
2012
70%
2014
2016
60% 2018
Next Year
50%
Design Projects
40%
30%
20%
10%
0%
FPGA Design Language Adoption
* Multiple answers possible
ASIC: Verification Language Adoption Next Twelve Months
90%
2012
80% 2014
2016
70% 2018
Next Year
60%
Design Projects
50%
40%
30%
20%
10%
0%
VHDL Verilog Vera System C SystemVerilog Specman e C/C++ OTHER Testbench
ASIC Verification Language Adoption ** Multiple answers possible
ASIC: SystemVerilog Adoption wrt Design Size
90%
79%
80% < 1M
73% 1M-40M
70% >40M
60%
Design Projects
50%
45%
40%
30%
20%
10%
0%
SystemVerilog
2018 SystemVerilog Adoption wrt Design Size

FPGA: Verification Language Adoption
80%
2012
70%
2014
2016
60% 2018
Next Year
50%
Design Projects
40%
30%
20%
10%
0%
FPGA Design Language Adoption
ASIC: Assertion Language Adoption Next Twelve Months
90%
2014
80%
2016
70%
2018
60%
Next Year
Design Projects
50%
40%
30%
20%
10%
0%
Accellera Open Verification Library SystemVerilog Assertions (SVA) PSL Other
(OVL)
ASIC Assertion Language Adoption * Multiple answers possible
FPGA: Assertion Language Adoption Next Twelve Months
60%
2014
50% 2016
2016
40%
Next Year
30%
20%
10%
0%
Accellera Open Verification Library SystemVerilog Assertions (SVA) PSL Other
(OVL)
FPGA Assertion Language Adoption * Multiple answers possible
Section F.
CLOCKING AND POWER

ASIC: Number of Asynchronous Clock Domain
35%
2012
2014
30% 2016
2018
25%
Design Projects
20%
15%
10%
5%
0%
1 2 3--4 5--10 11--20 21--30 31--50 >50
ASIC: Number of Asynchronous Clock Domain
FPGA: Number of Asynchronous Clock Domain
50%
2012
45% 2014
2016
40% 2018
35%
Design Projects
30%
25%
20%
15%
10%
5%
0%
1 2 3--4 5--10 11--20 21--30 31--50 >50
FPGA: Number of Asynchronous Clock Domain
ASIC: Actively Manage Power
100%
28% 28% 29%

80% 41% 38% 38%
Actively Manage Power

Design Projects
60%
No Power Management
40%
72% 72% 71%
59% 62% 62%
20%
0%
2007 2010 2012 2014 2016 2018
Designs that Actively Manage Power
ASIC: Power Management Features Verified
Hypervisor/OS control of power
management
Application-level power
management
Operation in each system power

state
Interactions between power

domains 2012
Hardware power control 2018
sequence generation
Transitions between system

power states
Power domain state

reset/restoration
Power domain power

down/power up
0% 10% 20% 30% 40% 50% 60% 70% 80%

Design Projects
ASIC: Notation to Specify Power Intent
40%
2012
35% 2014
2016
30%
2018
25%
Design Projects
20%
15%
10%
5%
0%
CPF 1.0 CPF 1.1 CPF 2.0 UPF 1.0 (Accellera) UPF 2.x (IEEE 1801) UPF 3.0 Internal / Proprietary
ASIC: Notation to Specify Power Intent
Section G.
SIMULATION METHODOLOGIES

ASIC: Methodologies and Testbench Base-Class Libraries
80%
2014
2016
70%
2018
60% Next Year
50%
Design Projects
40%
30%
20%
10%
0%
Accellera UVM Accellera OVM Mentor AVM Synopsys VMM Synopsys RVM Cadence eRM Cadence URM OSVVM UVVM None/Other
Portable
Stimulus ASIC: Methodologies and Testbench Base-Class Libraries
FPGA: Methodologies and Testbench Base-Class Libraries
60%
2014
2016
50% 2018
Next Year
40%
Design Projects
30%
20%
10%
0%
Accellera UVM Accellera Portable OVM Mentor AVM OSVVM UVVM None/Other
Stimulus
FPGA: Methodologies and Testbench Base-Class Libraries
Section H.
VERIFICATION TECHNIQUES

ASIC: Formal Technology Adoption
40%
2012
35% 2018
30%
25%
Design Projects
20%
15%
10%
5%
0%
Formal property checking Automatic formal verification
ASIC: Adoption of Static Techniques
FPGA: Formal Technology Adoption
25% 2012
2014
2016
20% 2018
15%
Design Projects
10%
5%
0%
Formal property checking Automatic formal verification
FPGA: Adoption of Static Techniques
ASIC: Adoption of Dynamic Techniques
Code coverage
Functional coverage
Assertions
2012
2014
2016
Constrained-Random 2018
0% 10% 20% 30% 40% 50% 60% 70% 80%
Design Projects
FPGA: Adoption of Dynamic Techniques
Code coverage
Functional coverage
Assertions
2014
2016
2018
Constrained-Random
0% 10% 20% 30% 40% 50% 60% 70% 80%
Design Projects
ASIC: Signoff Criteria
When all tests documented in the verification plan are complete and pass
When the project plan schedule says sign-off, assuming verification ok
When the emulated or prototyped design is working in-lab
When code or functional coverage says we have achieved our target
When we can no-longer think of any more tests to write
When the rate of bugs found per week drops below a specified goal
2014
When the project plan says sign-off, regardless of status
2018
Other
0% 10% 20% 30% 40%
ASIC Design Projects * Multiple answers possible

FPGA: Signoff Criteria
When all tests documented in the verification plan are complete and pass
When the project plan schedule says sign-off, assuming verification ok
When the emulated or prototyped design is working in-lab
When code or functional coverage says we have achieved our target
When we can no-longer think of any more tests to write
When the rate of bugs found per week drops below a specified goal
When the project plan says sign-off, regardless of status

2014
Other 2018
0% 10% 20% 30% 40% 50% 60%
FPGA Design Projects * Multiple answers possible

Section I.
VERIFICATION RESULTS

ASIC: Completion to Project's Original Schedule
40%
2012: 67% Behind Schedule 2012

30%
2016: 69% Behind Schedule
2016
2018
Design Projects
20%
10%
0%
More than 10% 10% EARLY ON-SCHEDULE 10% BEHIND 20% 30% 40% 50%
EARLY SCHEDULE
Actual ASIC design completion compared to project's original schedule
FPGA: Completion to Project's Original Schedule
40%

2014
30% 2016: 65% Behind Schedule
2018
Design Projects
20%
10%
0%
More than 10% 10% EARLY ON-SCHEDULE 10% BEHIND 20% 30% 40% 50% >50% BEHIND
EARLY SCHEDULE SCHEDULE
Actual FPGA design completion compared to project's original schedule
ASIC: Number of Required Spins Before Production
50% 2012
2014
2016
40%
2018
Design Projects
30%
20%
10%
0%
1 (FIRST SILICON 2 3 4 5 6 7 SPINS or MORE
SUCCESS)
Number of Required ASIC Spins Before Production
FPGA: Non-trivial Bug Escapes into Production
35% 84% of FPGA design projects 2016
have non-trivial bugs escape 2018

30% into production
25%
Design Projects
20%
15%
10%
5%
0%
0 1 2 3 4 5 6 or More
Non-trivial FPGA Bug Escapes into Production
2018 FPGA Verification Technique Adoption and Bug Escapes
61%
Code Coverage
77%
57%
Functional Coverage
65%
46%
Assertions
59%
Bug Escapes
30% No Bug Escapes
Constrained-Random
37%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Percentage Adoption
FPGA: FPGA Safety Critical Designs with Production Bug Escapes
Percentage of FPGA Projects Working on Safety Critical Designs with Bug Escapes to Production
15%
No Bug Escapes into Production
Bug Escapes into Production
85%
ASIC: Type of Flaws Contributing to Respin
60%
2012
2014
50%
2016
2018
40%
Design Projects
30%
20%
10%
0%
LOGIC OR CLOCKING TUNING ANALOG CROSSTALK POWER MIXED-SIGNAL YIELD OR TIMING – PATH FIRMWARE TIMING – PATH IR DROPS OTHER
FUNCTIONAL CIRCUIT CONSUMPTION INTERFACE RELIABILITY TOO SLOW TOO FAST
Type of ASIC Flaws Contributing to Respin * Multiple answers possible

FPGA: Type of Flaws Contributing to a Production Issue
60%
2012
2014
50%
2016
2018
40%
Design Projects
30%
20%
10%
0%
LOGIC OR CLOCKING CROSSTALK POWER MIXED-SIGNAL TIMING – PATH FIRMWARE TIMING – PATH OTHER
FUNCTIONAL CONSUMPTION INTERFACE TOO SLOW TOO FAST
Type of FPGA Flaws Contributing to a Production Issue

ASIC vs FPGA Flaws
50% FPGA
ASIC
40%
Design Projects
30%
20%
10%
0%
LOGIC OR FUNCTIONAL CLOCKING
ASIC vs FPGA Flaws
ASIC: Root Cause of Functional Flaws
80%
2012
70% 2014
2016
60%
2018
50%
Design Projects
40%
30%
20%
10%
0%
DESIGN ERROR CHANGES IN SPECIFICATION INCORRECT or INCOMPLETE FLAW IN INTERNAL REUSED FLAW IN EXTERNAL IP BLOCK or
SPECIFICATION BLOCK, CELL, MEGACELL or IP TESTBENCH
Root Cause of ASIC Functional Flaws * Multiple answers possible

FPGA: Root Cause of Functional Flaws
90%
2012
80% 2014
70% 2016
2018
60%
Design Projects
50%
40%
30%
20%
10%
0%
DESIGN ERROR CHANGES IN SPECIFICATION INCORRECT or INCOMPLETE FLAW IN INTERNAL REUSED FLAW IN EXTERNAL IP BLOCK or
SPECIFICATION BLOCK, CELL, MEGACELL or IP TESTBENCH
Root Cause of FPGA Functional Flaws * Multiple answers possible

ASIC: Biggest Challenge
CREATING SUFFICIENT TESTS TO VERIFY THE DESIGN (Coverage
Closure)
KNOWING MY VERIFICATION COVERAGE
MANAGING THE VERIFICATION PROCESS
TIME TO ISOLATE AND RESOLVE A BUG
TIME TO DISCOVER THE NEXT BUG

2012
2014
DEFINING APPROPRIATE COVERAGE METRICS 2016
2018
Other
0% 5% 10% 15% 20% 25% 30% 35% 40%
Design Projects
FPGA: Biggest Challenge
CREATING SUFFICIENT TESTS TO VERIFY THE DESIGN (Coverage
Closure)
KNOWING MY VERIFICATION COVERAGE
MANAGING THE VERIFICATION PROCESS
TIME TO ISOLATE AND RESOLVE A BUG
TIME TO DISCOVER THE NEXT BUG

2016
2018
DEFINING APPROPRIATE COVERAGE METRICS
Other
0% 10% 20% 30% 40% 50%
Design Projects
Section J.
CONCLUSIONS & MORE INFORMATION

Conclusions
 IC/ASIC projects are mature in their processes
 FPGA projects are being forced to mature their processes
 FPGA projects have a long way to go to prevent bug

escapes into production

More Information
 A video of this web seminar will be available out on the Verification
Academy in a few weeks
 My Verification Horizon Blog has all the data from previous studies,
and we are in the process of releasing this years study
 We are planning papers and presentations based on this study at

various conferences and seminars this next year
www.mentor.com

2018 Wilson Research Group: Functional Verification Study

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

2018 Wilson Research Group: Functional Verification Study

Caricato da

Copyright:

Formati disponibili

2018 Wilson Research Group

Functional Verification Study

It was the best of times, it was the worst of times...

© Mentor Graphics Corporation

Source: Statistics MRC 2018

Data Processing Automotive Mil/Aero

Consumer Telecommunications Industrial

35% 84% of FPGA design projects 2016

have non-trivial bugs escape 2018

 Sample frame consisted of 1205 participants

 Confidence interval 95%

ABOUT PARTICIPANT’S COMPANY

© Mentor Graphics Corporation

0% 5% 10% 15% 20%

0% 5% 10% 15% 20%

0% 5% 10% 15% 20% 25%

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

System architect FPGA

CAD support engineer

0% 5% 10% 15% 20% 25% 30% 35%

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

0% 5% 10% 15% 20% 25% 30% 35% 40%

FPGA Prototyping Engineer

Post-Silicon Validation Engineer

0% 5% 10% 15% 20% 25% 30% 35% 40%

0% 5% 10% 15% 20% 25% 30% 35% 40%

PARTICIPANT’S CURRENT DESIGN

© Mentor Graphics Corporation

High-Performance SoC FPGA

0% 5% 10% 15% 20% 25% 30%

0% 10% 20% 30% 40% 50% 60%

Current High-Performance SoC FPGA Type Being Implemented by Project

Gates of Logic and Datapath Excluding Memories

Gates of Logic and Datapath Excluding Memories

Geometry Feature Size

ASIC: Mean Number of Embedded Microprocessors by Design Size

64% of designs contain embedded processors 2016

• 64% of FPGA designs contain 1 or embedded processors

ASIC Design Projects

FPGA Design Projects

ASIC: Mean Number of Embedded DSP Cores by Design Size

IEC61511 - Process Industry

ISO25119 - Agriculture &…

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Safety Critical ASIC Design Projects

IEC61511 - Process Industry

ISO25119 - Agriculture &…

0% 5% 10% 15% 20% 25% 30% 35%

Safety Critical FPGA Design Projects

IEC61511 - Process Industry

IEC60601 - Medical FPGA

ISO25119 - Agriculture &…

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Safety Critical Design Projects

© Mentor Graphics Corporation

6.0 • Design Engineers growing at a 3.79% CAGR

13% Creating Test and Running Simulation

14% Creating Test and Running Simulation

Greater than 24 months

0% 5% 10% 15% 20% 25% 30%

2018 ASIC Design Projects