Several aspects of EXEC reflect its need to support multiprocessor as well as uniprocessor configurations.
2.14.1 SINGLE-THREADING
COS was originally developed on a uniprocessor system. Many code
sequences reflect the assumption that only one processor is active by how they access and update tables.
Rather than locating and changing all explicit and implicit assumptions regarding process synchronization within COS, the much less timeconsuming decision was made, that COS should run in only one CPU at anyone time.
On the CRAY X-MP mainframe, the hardware semaphore registers are used by EXEC to ensure that only one CPU is active in either EXEC or in the STP area. When one CPU is found to be in COS, the other CPU waits for the other CPU to leave the operating system; the code that accomplishes this begins at label LOCKOS in EXEC and extends through the end of the SYSWAIT subroutine.
Because SYSWAIT executes in monitor mode with external interrupts
disabled, and because when one CPU is in monitor mode no I/O interrupts are posted to another CPU of a CRAY X-MP system, SYSWAIT polls for I/O interrupts. When an I/O interrupt is found and when the other CPU is not in EXEC, SYSWAIT sends an interprocessor message and interrupts the other CPU.
SM-0040 2-94 C
I
EXEC MULTIPROCESSOR CONSIDERATIONS
2.14.2 SEMAPHORE USAGE
Management of CRAY X-MP cluster registers is the responsibility of three entities in the COS environment: EXEC, which manages the system cluster, COS locks, and saves and restores user cluster registers; JSH, which assigns nonsystem clusters to user jobs (and the contained user tasks) and directs EXEC to load or save user clusters; and to user-mode code, both user programs and library subroutines.
EXEC saves and restores user cluster registers at the direction of JSH.
As an aid to the library scheduler, EXEC also clears semaphore SM@BWAIT (SMOO) on every exchange to a user task. EXEC does not modify any other registers in user clusters.
Several semaphore registers are used within EXEC for interprocessor communication and coordination:
• SM@ALOCK - referred to as the active lock - is the master lock within EXEC. It is a short-term lock, used when attempting to gain access to other longer-term locks. Code sequences involving SM@ALOCK always follow the general pattern:
WAIT$SET ALaCK Test and set master lock GETSM xxxx Test secondary lock
$IF SO,MI If secondary lock is busy CLRSM ALaCK Release master lock
J delay Exit to do something else, or try again
$ENDIF
SETSM xxxx Set secondary lock CLRSM ALOCK Release master lock
• SM@PLOCK - called the passive lock - is used within EXEC to ensure single-threading of COS. The only portions of COS that are not single-threaded are small portions of EXEC: between labels EN and LOCKOS, and subroutines called by SYSWAIT (DEBUG, and IPCPU).
When SM@PLOCK is set, one X-MP CPU is active in COS.
• SM@EXEC flags for the SYSWAIT subroutine. When SM@EXEC is set, a CPU is in the single-threaded section of EXEC; when clear, then no CPU is in EXEC. SM@PLOCK is always set when SM@EXEC is set,
though the reverse is not true.
• SM@IPRQ controls access to the Interprocessor Request Table
(B@IPRQ). Writes into the table can only take place after issuing a WAIT$SET IPRQ instruction. This lock is needed because writes into the IPRQ table can take place outside of code protected by SM@PLOCK.
SM-0040 2-95 C
I
MULTIPROCESSOR CONSIDERATIONS EDC
• SM@DEBLK ensures single-threading in the DEBUG subroutine. This semaphore will always be set in the history trace entries for X-MP systems. This lock is needed because DEBUG calls can be made from outside of code protected by SM@PLOCK.
• SM@BWAIT flags for the memory error correction code. When set, SM@BWAIT indicates that a CPU is in the SYSWAIT subroutine.
Memory error correction uses this flag to determine that the other CPU is parked in EXEC so that memory correction can be safely done. SM@BWAIT (in the system cluster) is set and cleared only in the SYSWAIT subroutine.
2.14.3 INTERPROCESSOR COMMUNICATIONS
The RCP and DCP Executive requests (connect and disconnect user task) are issued by JSH to associate and disassociate a user task from the
specified cPU. Because the requests are CPU-specific, EXEC provides JSH with a mechanism (PSWITCH) for switching between physical CPUs.
The PSWITCH Executive request, available only to JSH, performs the necessary interprocessor communication. When JSH calls PSWITCH, EXEC suspends the caller and verifies that the caller was indeed JSH.
EXEC, in addition to accomodating JSH, also needs the ability to switch physical CPUs. EXEC causes the other CPU to enter EXEC either because an
I/O interrupt is found while in SYSWAIT, or because memory error correction needs to have all user-mode exchange packages in memory
(having all CPUs in EXEC guarantees no user-mode exchange package is active).
Processor switching is accomplished through a mechanism called
interprocessor communication. Interprocessor communication takes place through messages. Interprocessor messages are currently of two types:
• Processor switch messages to allow JSH to use the other CPU
• No-op messages to allow EXEC to use the other CPU EXEC routine IPCPU places the interprocessor message in the
Interprocessor Request Table (IPRQ) and issues an interprocessor
interrupt (IP 1) instruction. Interprocessor interrupts, while used in sending messages, are basically ignored by EXEC.
When the other CPU enters EXEC, EXEC receives the message. IPREQST processes processor switch messages and schedules JSH in the receiving CPU.
SM-0040 2-96 C
I
EXEC EXEC-SPECIFIC MACROS
2.14.4 PROCESSOR WORKING STORAGE AREA (PWS)
The processor working storage area (PWS) contains data specific to each CPU, including:
• Addresses and fields associated with a connected user task, including the user task exchange package, if any.
• Idle task exchange package and BOO register save areas.
• Memory error correction exchange package.
• Statistics and timing counters, both cumulative and in last statistics interval.
• A word indicating which software process is executing in the CPU in question (USER, EXEC, MEM-COR, or system task) •
The GETPW macro can be used to determine the PWS address of the current CPU (see section 2.15.4) •