A Cosimulation Analyzer to Validate an SCI-to-PCI Bridge Design
Von der Fakultät für Elektrotechnik der
Rheinisch-Westfälischen Technischen Hochschule Aachen zur Erlangung des akademischen Grades eines Doktors der
Ingenieurwissenschaften genehmigte Dissertation
vorgelegt von
Diplom-Ingenieur Johann Friedrich Fischer aus Norderney
Berichter: Univ.-Prof. Dr. phil. (Uni Wien) Heinrich Kurz Univ.-Prof. Dr.-Ing. Tobias Noll
Tag der mündlichen Prüfung: 26. Januar 1999
D 82 (Diss. RWTH Aachen)
D 82 (Diss. RWTH Aachen)
Shaker Verlag Aachen 1999
Berichte aus der Kommunikationstechnik
Johann F. Fischer
A Cosimulation Analyzer to Validate an SCI-to-PCI Bridge Design
.
Die Deutsche Bibliothek - CIP-Einheitsaufnahme Fischer, Johann F.:
A Cosimulation Analyzer to Validate an SCI-to-PCI Bridge Design / Johann F.
Fischer. - Als Ms. gedr. - Aachen : Shaker, 1999
(Berichte aus der Kommunikationstechnik) Zugl.: Aachen, Techn. Hochsch., Diss., 1999 ISBN 3-8265-4912-0
Copyright Shaker Verlag 1999
Alle Rechte, auch das des auszugsweisen Nachdruckes, der auszugsweisen oder vollständigen Wiedergabe, der Speicherung in Datenverarbeitungs- anlagen und der Übersetzung, vorbehalten.
Als Manuskript gedruckt. Printed in Germany.
ISBN 3-8265-4912-0 ISSN 0945-0823
Shaker Verlag GmbH Postfach 1290 52013 Aachen Telefon: 02407 / 95 96 - 0 Telefax: 02407 / 95 96 - 9
Internet: www.shaker.de eMail: info@shaker.de
.
Für Birgit, Keno und Taale
vii Parts of this work have been published:
1. J. Fischer, C. Müller, H. Kurz, "Testing and Analysis of a Complex Logic Design Using a Co- Simulation Concept". Proceedings of the 1st Workshop on Design , Test and Applications, pp.
93-96, Dubrovnik, Croatia, June 8-10, 1998
2. J. Fischer, C. Müller, H. Kurz, "A Co-Simulation Concept for an Efficient Analysis of Complex Logic Designs". Proceedings of the 8th International Workshop on Field Programmable Logic and Applications (Lecture Notes in Computer Science 1482), pp. 495-499, Tallinn, Estonia, August 31 - September 3, Springer Verlag, Heidelberg, 1998
ix
Acknowledgements
The presented work was conducted at the Institut für Halbleitertechnik II (IHT II -- Institute of Semiconductor Electronics II) at the RWTH Aachen (Technical University of Aachen/Germany).
The GigaNet project has been carried out in a cooperation between the IHT II and the Gesellschaft für Angewandte Mikro- und Optoelektronik mbH, Aachen (AMO GmbH).
I am especially grateful to my supervisor Prof. Dr. H. Kurz for his interest in my work and the opportunity to conduct it at his institute. His willingness to accept the fact that I was also still employed in industry during this time while allowing me the required independence and flexibility has made this effort possible.
Special thanks to Prof. Dr.-Ing. T. Noll for willingly accepting the Korreferat (second examination) and the valuable discussions.
Dr.-Ing. Thomas Voß has provided me with a lot of helpful hints, background and discussions, which I deeply appreciate. It was an enlightening experience to work with him.
Dr.-Ing. Clemens Müller has been a close colleague always willing to discuss the details of reprogrammable logic and to share ideas, both inside and outside our work. This includes the soccer matches whenever we had consumed enough Snikers bars. Many thanks.
Thanks also to the colleagues at IHT II and AMO who helped to make the passed four years a success.
Special thanks are due to Eberhardt Wegener, who offered to read the draft of my thesis. Even though he did not know what was coming his way, he still agreed when I took him up on his offer.
He provided me with a lot of good suggestions being the uninvolved external reviewer. Thanks a lot.
My approach to engaging in a Ph.D. program has been somewhat unusual by German standards.
After graduating from university with a Diplom (master degree), I decided to find out "what's out there" in industry. It was only after 2 ½ years that I returned to the Alma Mater. It took the exceptional support of my former manager at 3M Laboratories (Europe) GmbH, Mr. Georg Ruß, which I gratefully acknowledge. Because of this support I was able to pursue this endeavor while staying part-time employed by 3M Eurolab and later Imation Germany. G. Ruß has proved to live the visions of 3M especially that employees are 3M's most valued resources, allowing me the same flexibility and independence that I was granted by Prof. Kurz. It has been a pleasure to work for him. I would also like to acknowledge the support and good advice through the years of Ms. U. Dravenau-Otto and Mr. W. Kock, both with 3M Eurolab's personnel department at the time, when I started the program. Both supported the idea and laid a contractual foundation that allowed me to finish my degree even after the spin-off into Imation Corp. in July 1996.
x
Support from colleagues and supervisors was required to get to the point in my education where I am now, but it is the foundation, love and support of my family that allowed me to do all this. My parents Friedrich and Rosemarie Fischer gave me and my sisters Anja and Ebba their values and appreciation for education that has culminated in this work. They have provided us with the roots as children and the wings when it was time to fly. One of these flights brought me to Topeka, Kansas, during my AFS exchange year where I now have a second family. My American parents Dr. Mark L. Morris Jr. and Dr. Bette M. Morris supported me all the way including their guidance and advice in choosing Colorado State University in Fort Collins, CO for my Fulbright year in college. Together with my American brothers and sister, Mark Lee, David, and Cynthia Lynn they have made me love life in the Midwest.
During the passed four years since 1995, I feel that I have accomplished a lot. I worked in industry, successfully pursued a Ph.D. degree and founded a family during this time. What more can I ask for. My wife Birgit, who married me in 1995, took much of the burden in running our family and keeping our son Keno (born 1995) and our daughter Taale (born 1997) happy. Her parents Dr. Kurt-Henry and Maria Mindermann have always been there during that time to help, which we gratefully appreciate. It was Birgit’s loving and caring that made these years especially wonderful and helped me through those gloomy moments that hit all of us at times. In many ways it feels like it is time to head towards new grounds yet another time in our lives. I am looking forward to that.
Keno and Taale may remember getting kicked out of our home office more than once lately when I needed time to work away. But they found plenty of opportunities to brighten my days emptying my book shelves when the door was not tightly closed or hitting the mice running over my computer screen.
xi
Table of Contents
ACKNOWLEDGEMENTS ... ix
TABLE OF CONTENTS ... xi
LIST OF FIGURES... xv
LIST OF TABLES ... xvii
1 INTRODUCTION ... 1
2 SYSTEM OVERVIEW ... 5
2.1COMPLEX LOGIC CIRCUITS... 5
2.1.1 The Cosimulation Paradigm ... 5
2.1.2 Validation Approaches for Complex Logic ... 6
2.1.3 Importance of System Environment for Design Validation... 8
2.2 GIGANET HARDWARE... 9
2.2.1 Hardware Overview ... 9
2.2.2 Hardware Development Process ... 11
2.2.3 Fields of Application for GigaNet Hardware ... 13
2.2.3.1 Workstation Cluster Environments ... 13
2.2.3.2 Real-time Environments ... 14
2.2.3.3 Data Distribution System Environments ... 15
3 COSIMA RELATED STANDARDS... 17
3.1 PERIPHERAL COMPONENT INTERCONNECT... 17
3.2 IEEE 1596 - SCALABLE COHERENT INTERFACE BASE STANDARD... 21
3.2.1 SCI Protocol Overview... 21
3.2.2 Ringlet Priority... 27
3.2.3 Bandwidth Partitioning... 30
3.2.4 Fair Bandwidth Allocation (Arbitration Protocol) ... 31
3.2.5 Queue Allocation... 32
3.2.5.1 Queue Reservation... 32
3.2.5.2 Allocation Count Maintenance... 33
3.2.5.3 Queue Selection... 33
3.2.6 SCI Interconnect Initialization... 34
3.2.6.1 Address Initialization... 34
3.2.6.2 Ringlet Initialization ... 35
3.2.7 Additional SCI Protocol Aspects ... 36
3.3 IEEE 1596.6 - SCI/RT... 37
3.3.1 SCI/RT Preemptive Priority Queue Protocol ... 39
3.3.2 SCI/RT Train Protocol ... 41
3.3.3 SCI/RT Directed Flow Control Protocol... 43
3.3.4 SCI/RT 2 Bit Priority Protocol ... 45
3.4 IEEE 2100 - SERIALEXPRESS... 49
xii
3.4.1 SerialExpress Overview ...49
3.4.2 SerialExpress Switches and Bridges...53
3.4.3 SerialExpress Queues and Queue Allocation...54
3.4.4 SerialExpress Protocols...54
3.4.4.1 Ringlet Administration ...54
3.4.4.2 Ringlet Arbitration ...55
3.4.4.3 Isochronous Traffic Flow Control ...57
3.4.4.4 Ringlet Initialization and Reset...58
3.5 SCI EXTENSIONS SUMMARY...59
4 COSIMA DESIGN AND IMPLEMENTATION ...61
4.1 REQUIREMENT ANALYSIS AND SPECIFICATION...61
4.2 DESIGN...65
4.3 IMPLEMENTATION...71
4.3.1 Choosing a Simulation Environment...71
4.3.2 Transaction Coordinator...73
4.3.3 SCI Protocol Handler Subsystem ...74
4.3.3.1 SCI Linc State Machine...75
4.3.3.2 SCI Error Generator State Machine ...78
4.3.4 PCI Protocol Handler Subsystem ...80
4.3.4.1 PCI Bus State Machine...80
4.3.4.2 PCI Arbiter State Machine ...87
4.3.5 Cosimulation Communication Interface ...88
4.3.6 Data Logger ...91
4.3.7 Configuration Handler and Error & Output Logger...94
4.3.7.1 CoSimA Configuration File Structure ...95
4.3.7.2 Test Case Configuration File Structure...96
4.4 COSIMA VALIDATION...97
5 COSIMA SIMULATION RESULTS...99
5.1 COMMUNICATION INTERFACE...99
5.2 PROTOCOL IMPLEMENTATION VALIDATION RESULTS...100
5.2.1 PCI Write Cycle...100
5.2.2 PCI Read Cycle...103
5.2.3 SCI ReadSB Cycle...105
5.2.4 SCI WriteSB Cycle ...107
5.2.5 Error Test Case Simulation Results...108
5.2.6 Test Result Summary...109
5.3 PERFORMANCE SIMULATION RESULTS...110
5.3.1 Cosimulation References with VhdlSim...110
5.3.2 Cosimulation Results with GigaNet Design ...113
6 CONCLUSIONS ...115
7 OUTLOOK...117
APPENDIX A COMPARISON OF THE SCI BASED STANDARDS ...123
A.1 DESIGN OBJECTIVES...124
A.2 FEATURES...126
A.3 PHYSICAL LAYER...129
A.4 TRANSACTION TYPES...132
A.5 PROTOCOLS...134
xiii
APPENDIX B IEEE 1596 SCI C CODE ... 141
B.1 CHANGE LOG... 142
B.2 SCI C CODE SOURCE FILES... 143
B.2.1 C Code Files and Switch Settings ... 144
B.2.2 Compiler Macros ... 144
B.2.3 Compile Time Options Used for Simulation Runs ... 145
B.3 PROGRAM DOCUMENTATION... 145
B.3.1 Configuration Information ... 145
B.3.1.1 Command Line Switches for the Executable ... 145
B.3.1.2 Topology Setup ... 146
B.3.2 General Remarks ... 146
B.3.3 Program Flow ... 146
B.3.4 Thread Handling ... 147
B.3.4.1 Thread Scheduling ... 147
B.3.4.2 Context Switching... 148
B.3.5 Time Variables ... 149
B.3.6 Transaction Data Exchange ... 150
B.3.6.1 Intra-node Communication... 150
B.3.6.2 Inter-node Communication... 153
B.3.6.3 Additional Information... 153
B.3.7 Symbol Flow Through Linc Chip ... 157
B.3.8 Signal Lines ... 165
B.3.9 Chip States... 166
B.3.9.1 Power States... 166
B.3.9.2 Run States ... 167
B.3.9.3 Core States ... 167
B.3.9.4 Linc States ... 167
APPENDIX C COSIMA DOCUMENTATION ... 169
C.1 DESIGN DOCUMENTATION... 169
C.1.1 CoSimA Design Objects ... 170
C.1.2 List of Contracts ... 183
C.2 TEST CASES... 184
APPENDIX D SCI RELATED WORK ... 191
D.1 GENERAL SWITCHING AND ROUTING... 192
D.2 SWITCHING AND ROUTING IN SCI INTERCONNECTS... 195
D.3 DIFFERENT SCI INTERCONNECT TOPOLOGIES... 196
BIBLIOGRAPHY ... 199
CURRICULUM VITAE ... 205
xv
List of Figures
Figure 2-1 GigaNet Hardware Overview ... 10
Figure 2-2 Hardware Development Process... 12
Figure 3-1 PCI System Block Diagram... 18
Figure 3-2 PCI Pin List... 20
Figure 3-3 PCI Write Cycle... 20
Figure 3-4 SCI Packet Format ... 22
Figure 3-5 SCI Split Response Transaction ... 23
Figure 3-6 SCI Node Linc Interface Structure ... 24
Figure 3-7 SCI Idle Symbol... 25
Figure 3-8 SCI Flag Bit Packet Delimiting ... 26
Figure 3-9 Queue Allocation State Machines ... 32
Figure 3-10 SCI Flow Control Protocol Extension ... 36
Figure 3-11 SCI/RT PPQ Idle Symbol... 40
Figure 3-12 SCI/RT TRAIN Protocol Idle Symbol... 42
Figure 3-13 SCI/RT DFC Protocol Idle Symbol ... 44
Figure 3-14 SCI/RT 2-Bit Priority Protocol Idle Symbol... 48
Figure 3-15 SerialExpress Proficient Node Structure (2 attachments) ... 50
Figure 3-16 SerialExpress Node Daisy Chains ... 51
Figure 3-17 Write64 Data Selection ... 52
Figure 3-18 SerialExpress Idle Symbol ... 53
Figure 3-19 SCI Node Linc Functional Blocks ... 57
Figure 4-1 Cosimulation Concept Block Diagram ... 62
Figure 4-2 Hierarchy Graph... 66
Figure 4-3 Client-Server Contract ... 66
Figure 4-4 Collaboration Graphs ... 67
Figure 4-5 CoSimA Design Collaboration Graph (detailed) ... 68
Figure 4-6 SCI Transaction Manager Simulation Loop ... 75
Figure 4-7 CoSimA State Machine Inheritance Graph... 75
Figure 4-8 SCI Communication Symbol Sequence... 76
Figure 4-9 Test Case "Delete" State Transitions ... 79
Figure 4-10 PCI Read Cycle with Symbol Mapping... 81
Figure 4-11 PCI Bus Master Cycle State Transition Diagram... 83
Figure 4-12 PCI Bus Target Cycle State Transition Diagram ... 84
Figure 4-13 PCI Arbiter State Transition Diagram ... 85
Figure 4-14 PCI Communication Symbol Sequence... 87
Figure 4-15 CoSimA Synchronized Files Communication Mechanism ... 88
Figure 4-16 CoSimA Communication Exchange Handshake... 89
Figure 4-17 Deadlock in Communication Interface ... 90
Figure 4-18 Deadlock Avoidance in Communication Interface ... 90
Figure 4-19 Packet Matching Scenarios... 93
xvi
Figure 4-20 CoSimA Configuration File - Selected Parameters ...96
Figure 4-21 CoSimA Test Case Configuration File - Selected Parameters ...97
Figure 5-1 PCI Write Cycle - 1 DWORD Transferred...100
Figure 5-2 PCI Write Cycle - CoSimA Trace...101
Figure 5-3 PCI Write Cycle - 3 DWORDs Transferred ...102
Figure 5-4 PCI Write Cycle - Target Abort ...102
Figure 5-5 PCI Write Cycle Address Error...103
Figure 5-6 PCI Read - Target Abort ...104
Figure 5-7 PCI Read Cycle - Beyond 16 Byte Boundary...104
Figure 5-8 SCI Response - Premature Issuing of Echo Packet...105
Figure 5-9 WriteSB - Incorrect TID in Echo Packet...106
Figure 5-10 PCI Cycle in Response to SCI WriteSB...106
Figure 5-11 SCI Bypass Traffic Corruption ...108
Figure 5-12 PCI Read Transactions with VhdlSim...111
Figure 5-13 PCI Write Transactions with VhdlSim...111
Figure 5-14 SCI ReadSB Transactions with VhdlSim ...112
Figure 5-15 SCI WriteSB Transaction with VhdlSim...112
Figure 5-16 PCI Write - Heavy Traffic Load ...113
Figure 5-17 PCI Write - Light Traffic Load ...114
Figure 5-18 PCI Write - Mixed Traffic Load ...114
Figure 7-1 Object Models: CORBA / DCOM ...118
Figure B-1 IEEE SCI C Code Pointer Data Structures ...147
Figure B-2 Node-local Communication Queues...152
Figure B-3 Cloud ioQueue Naming Convention ...158
Figure C-1 State Machine Hierarchy Graph ...181
Figure C-2 CoSimA Design Collaboration Graph (Subsystems) ...181
Figure C-3 Subsystem SCI Protocol Handler Collaboration Graph ...182
Figure C-4 Subsystem PCI Protocol Handler Collaboration Graph ...182
Figure C-5 Subsystem File I/O Collaboration Graph...182
Figure D-1 Interconnect Topologies...192
xvii
List of Tables
Table 3-1 PCI Bus Commands ... 19
Table 3-2 SCI and its Extensions - A Short Comparison ... 60
Table 4-1 Test Case Classification... 70
Table 4-2 Mapping of PCI Signal Line States to CoSimA Communication Symbols ... 82
Table 4-3 PCI Bus State Machine Entry and Leaving Actions... 87
Table 5-1 Busy Looping Evaluation ... 99
Table 5-2 Simulation Run Result Statistics... 114
Table A-1 Comparison of SCI Based Standards (Design Objectives) ... 125
Table A-2 Comparison of SCI Based Standards (Features) ... 128
Table A-3 Comparison of SCI Based Standards (Physical Layer) ... 131
Table A-4 Comparison of SCI Based Standards (Transaction Types)... 133
Table A-5 Comparison of SCI Based Standards (Protocols) ... 140
Table B-1 Elasticity State Transition Diagram ... 160
Table B-2 Stripper Packet Parser State Transition Diagram... 163
