meI<D
Parallel ProgrammingThe Elan Widget Library is a high performance message passing library that was developed specifically for the CS-2. It provides a number of low level communi-cation constructs that can be used by developers of higher level message passing libraries (CSN, MPSC, PVM, PARMACS etc. are all built upon the Elan Widget library) or for application developers wishing to optimise communications per-formance.
The relationship between the Widget library and Meiko's higher level libraries means that Widget library functions may be embedded within most CS-2 appli-cations.
Key features of the Elan Widget library are:
• Global memory management.
• Process grouping.
• Access to Elan DMA engine offering high performance network transfers.
• Support for parallel file I/O.
• Channel and broadcast channels.
• Tagged message ports (TPORTs).
• Exception handling.
• Global synchronisation.
• Group reduction and exchange.
• C interface.
The Widget library memory management functions allow regions of memory to be allocated at the same virtual address on a number of processes. This is an im-portant feature of the Elan Widget library (and is used in the implementation of many Widget library communication functions). It allows processes to transfer data directly into the address space of remote processes without prior handshak-ing of buffer addresses.
67
68
A process group defines a number of processes that wish to cooperate in barriers, reductions, and communications. Processes may belong to more than one group, and group definitions may overlap.
Direct access to the Elan DMA engine is provided, allowing either local or re-mote store-to-store accesses. In many cases data transfer between processes will use one of the higher level message passing facilities: channels or tagged mes-sage ports (TPORTs). Channels provide a full duplex, non-blocking, unbuffered communication between a process pair; only one transmit and one receive may be active on a channel at anyone time. Broadcast channels are also supported al-lowing a process to broadcast to a contiguous range of processes using the Elan's broadcast functionality. Tagged message ports provide a more general communi-cation mechanism, offering communicommuni-cation between arbitrary processes using either blocking or non-blocking, buffered or unbuffered, communications, with message selection at the receiver by either user specified message tag or sender id.
Global reduction functions allow data that is distributed among the processes in a group to be combined according to some user supplied function. Global ex-change functions provide a mechanism for distributing data among a process group. Both facilities offer significantly higher performance than the equivalent sequence of communications and calculations, especially when the size of trans-fer is small (and start-up latencies are more significant).
For more information about this library see the Elan Widget Library, Meiko doc-ument number SlOO2-10MI04.
Example
The following example is a Widget library implementation of the CSN example described earlier. The processes communicate in a ring, each performing a simple addition (essentially a global reduction operation)
The application begins by initailising the base programming environment with a call to ew_baselnitO. This initialises both the ew_state and ew_base structures: the ew_state structure identifies each process's virtual process id and the number of processes in the application; the ew _base structure identifies the alloc region (global memory) and TPORT (tagged message port) parameters used in this example.
Sl002-10Ml17.02
mei<a
iinelude <sys/types.h>
ELAN_EVENT *RxEvent, *TxEvent;
int rx, tx, sum, i;
/* Intialise base environment */
ew_baselnit () :
me = ew_state.vPi /* My process id */
nproes = ew_state.nvp-l;
alloc = eW_base.alloc;
if(! (tport (EW_TPORT*) ew_allocate(alloc, EW_ALIGN,
ew_tportSize(ew_base.tport_nattn»»
fprintf(stderr, "Failed to allocate\n"):
exit(l):
5
Each process allocates its TPORT structure as a global object by using ew _ al-locateO. Because global objects exist at the same virtual address in all our processes a sending process can target a recipient TPORT without explicit hand-shaking of addresses.
meko
Parallel Programming 6970
Note that the TPORT is initialised with the default number of attention slots (cur-rently 4, as defined by ew _ baselni to). The attention slots detennines the maximum number outstanding communications that may be present on a TPORT at any time. All of Meiko 's current message passing libraries are initialised with this default.
Having initialised the TPORTs the processes barrier synchronise with a call to ew _ f g s yn
cO.
The sychronisation point ensures that no process is able to target a TPORT before its initialisation is complete. The synchronisation uses one of the groups defined during the base initialisation; the segGroup is a group of all process in the segment (i.e. all processes excluding the loader program)./* initialise TPORT */
ew_tportlnit(tport, ew_base.tport_nattn, me, ew_base.tport_smallmsg, ew_base.waitType, ew_base.dmaType);
/* Wait until every process has initialised its TPORT */
ew_fgsync(ew_base.segGroup);
printf("Sum = %d\n", sum);
Each process receives a non-blocking, unbuffered communication from one neighbour, and initiates a non-blocking unbuffered send to its other neighbour (conceptually the processes are connected in a ring). After nproc communica-tions each process has calculated the sum of the virtual process id's; process 0 displays the result.
5
TxEvent - ew_tportTxStart(tport, EW_TPORT_TXSYNC, right, tport, 0, (caddr_t) &tx, sizeof (tx) ) ; sum +- tx;
/* block for completion - not interested in envelope information */
ew_tportRxWait(RxEvent, NULL, NULL, NULL);
ew_tportTxWait(TxEvent);
sum += rx;
if (me == 0)
printf(ItSum %d\nl t, sum);
To compile a Widget library application you will need the following flags:
user@cs2: cc -0 tport -I/Opt/MEIKOcs2/inc1ude \ -L/opt/MEIKOcs2/1ib tport.c -1ew -1e1an
To run the program use prun:
user@cs2: prun -n4 -ppara11e1 widget Sum = 6