United States Patent [19]

[11] Patent Number:

Adams, III et a1.

[45]

[54] EXTRA STAGE CUBE [75] Inventors: George B. Adams, III; Howard J. [73] Assignee: [21]

Siegel, both of West Lafayette, 1nd.

of data streams comprises a cube comprising a plurality

Purdue Research Fmmda‘ion,

of stages sequentially numbered n—l through 0 and an

Lafayene, Ind.

extra stage n situated to proceed n— 1 of the cube.

Stages 11 through 0 comprise a plurality of data switch ing boxes. Each box includes a ?rst input port, a second

Dec_ 23’ 193;

input port, a ?rst output port, a second output port, and

3

apparatus for selectively coupling the ?rst input port to

Int. CL .............................................. ..

[2%] [ l

h 'e

0

at least one of the outpu‘ ports and thg second input port

41/

can

to at least one of the output ports. Extra stage n and

900 MS ‘F1 ’

stage 0 include apparatus for selectively bypassing stage

1e

n or stage 0 or neither stage n nor stage 0v The system

’

[56]

References Cited US. PATENT DOCUMENTS 3,794,983 4,011,545 4,060,795 4,090,174 4,247,892 4,301,443 4,414,685

Jun. 11, 1985

[57] ABSTRACT A multistage data routing system for routing a plurality

APPL N°~= 452,438

[22] Filed;

Date of Patent:

4,523,273

further comprises a plurality of links, each coupling a selected ouput port of a selected stage to a selected

input port of the next adjacent downstream stage. The

2/1974 Sabin ................................... .. 382/28 3/1977 Nadir ...................... .. 364/200 364/900 11/1977 Harumiya et a1. .. 364/900 5/1978 Van Voorhis .... .. . 364/200 V1981 Lawrence ......... .. 382/49 11/1981 Sternberg et a1. .. 11/1983 Sternberg ............................ .. 382/49

system also comprises a system control unit which cou

ples a selected input port of a data switching box of stage n to at least one of the ouput ports of the data

switching boxes of stage 0 and selectively simulta neously disables the selective bypassing apparatus of at least one of stage n and stage 0.

Primary Examiner—-Harvey E. Springborn 14 Claims, 11 Drawing Figures

Attorney, Agent, or Firm—Barnes & Thornburg

SCU

STAGE

3

2

US. Patent Jun. 11,1985

Sheet2 013

4,523,273

OUTP OUTP

23 H

T

456 F142 6

—1

STAGE

2

OUTP

FIG]

STAGE

3

U.S. Patent 1mm

Jun. 11, 1985

_

o1

Sheet 3 0f 3

“U

|'_' 40

,_.1o2

_01

'?|

‘|_|

—

lab

1|_H"l 3

- 0M

_2\ ¢|—|i D _ 2 ‘Pia-p 3

2i/ "13%| "ll! . 4-‘

7

7

.

'

STAG E

3

2

6

+='~ IHI'

L7 £| O

CUBE FUNCTION IM PLEMENTED 12

F169

STAGE

3

CUBE FUNCTION IMPLEMENTEDI Q

2

4,523,273

1

4,523,273

EXTRA STAGE CUBE

2

lying the partitioning of permutation networks," IEEE Trans. Comput., vol. C-29, pp. 791-801, September

The work which resulted in this invention was sup

1980) and distributed control through the use of routing tags (see Lawrie, supra) are available in the ESC.

ported by US. Air Force Office of Scienti?c Research,

Therefore, the ESC is a practical answer to the need for

Air Force Systems Command, under Grant AFOSR

reliable communications in parallel/distributed super systems.

78-3581 and the National Science Foundation under Grant ECS 80-16580. The Government has certain

rights in this invention. The demand for very high-speed processing coupled with falling hardware costs has made large-scale paral lel and distributed supercomputer systems both desir able and feasible. An important component of such

Multistage cube-type networks have been proposed for many supersystems. These include PASM (see H. J. Siegel, L. J. Siegel, F. C. Kemmerer, P. T. Mueller, Jr., 11. E. Smalley, Jr., and S. D. Smith, "PASM: A parti

tionable SIMD/MIMD system for image processing and pattern recognition,” IEEE Trans. Comput., vol. among the computation nodes and memories. Because 15 030, pp. 934-947, December 1981), PUMPS (see F. A. Briggs, K. Hwang, K. S. Fu, and M. C. Dubois, of system complexity, assuring high reliability is a sig "PUMPS architecture for pattern analysis and image ni?cant task. Thus, a crucial practical aspect of an inter database management,” in Proc. Pattern Recognition connection network used to meet system communica supersystems is a mechanism for information transfer

Image Processing Conf., August 1981, pp. 387-398), the tion needs is fault tolerance. Multistage cube-type networks such as the baseline 20 Ballistic Missile Defense Agency distributed processing test bed (see W. C. McDonald and J. M. Williams, “The (see, C. Wu and T. Feng, “On a class of multistage advanced data processing testbed,” in Proc. COMP interconnection networks,” IEEE Trans. Comput., vol. C-29, pp. 694-702, August 1980), delta (see J. R. Patel, “Performance of processor-memory interconnections

SAC, March 1978, pp. 346-351; and H. J. Siegel and R. J. McMillen, “The multistage cube: A versatile inter

for multiprocessors," IEEE Trans. Comput., vol. C-30, 25 connection network," Computer, vol. 14, pp. 65-76, December 1981), Ultracomputer (see A. Gottlieb, R. pp. 771-780, October 1981), Generalized Cube (see H. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, J. Siegel and S. D. Smith, “Study of multistage SIMD and M. Snir, The NYU ultracomputer—-a general-pur interconnection networks," in Proc. 5th Symp. Com pose parallel processor, Ultracomput. Note 32, Rep. put. Architecture, April 1978, pp. 223-229), indirect 034, July 1981, 33 pp), the Flow Model Processor of the binary n-cube (see M. C. Pease, III, “The indirect bi Numerical Aerodynamic Simulator (see G. H. Barnes, nary n-cube microprocessor array," IEEE Trans Com “Design and validation of a connection network for put., vol. C-26, pp. 458-473, May 1977), omega (see D. many-processor multiprocessor systems,“ in Proc. 1980 H. Lawrie, “Access and alignment of data in an array processor,” IEEE Trans. Comput., vol. C-24, pp. Int. Conf. Parallel Processing, August 1980, pp. 79-80), 1145-1155, December 1975), shuffle-exchange (see 5. 35 and data flow machines (see J. B. Dennis, G. A. Bough Thanawastien and V. P. Nelson, “Interference analysis ton, and C. K. C. Leung, “Building blocks for data flow of shuffle/exchange networks,” IEEE Trans. Comput., prototypes," in Proc. 7th Symp. Comput. Architecture, vol. C-30, pp. 545-556, August 1981), STARAN ?ip May 1980, pp. l-8). The ESC can be used in any of (see K. E. Batcher, “The ?ip network in STARAN,” in these systems to provide fault tolerance in addition to Proc. 1976 Int. Conf. Parallel Processing, August 1976, 40 the usual cube-type network communication capability. pp. 65-71), and SW-banyan (S=F=2) (see L. R. Goke The ESC can be used in various ways in different and G. J. Lipovski, “Banyan networks for partitioning computer systems. The ESC can be incorporated in the multimicroprocessor systems,” in Proc. 1st Symp. Com PASM and PUMPS supersystems. PASM, a partitiona put. Architecture, December 1973, pp. 21-28) have ble SIMD/MIMD machine being designed at Purdue been proposed for use in parallel/distributed systems. 45 University (see Siegel, Siegel, Kemmerer et a1, supra), is The Generalized Cube is representative of these net a dynamically recon?gurable multimicroprocessor sys works in that they are topologically equivalent to it (see tem using up to 1024 processing elements (processor/ H. J. Siegel, “Interconnection networks for SIMD ma memory pairs), or PE’s, to do image processing and chines," Computer, vol. 12, pp. 57-65, June 1979; Siegel pattern recognition tasks. In this context, the network and Smith, supra; and Wu and Feng, supra). The prob 50 would operate in a unidirectional, PE-to-PE packet lem with this topology is that there is only one path switched mode (see Siegel and McMillen, supra). The from a given network input to a given output. Thus, if PUMPS MIMD architecture (see Briggs, supra), also there is a fault on that path, no communication is possi under development at Purdue, consists of multiple pro ble. cessors with local memories which share special pur According to this invention, the Extra Stage Cube (ESC), a fault-tolerant network, is derived from the

Generalized Cube. The ESC is capable of operating in both SIMD and MIMD (see M. J. Flynn, “Very high speed computing systems," Proc. IEEE, vol. 54, pp. 1901-1909, December 1966) environments. The ESC comprises a Generalized Cube with an additional stage

pose peripheral processors, VLSI functional units, and a

common main memory. The network serves in a bidi

rectional, circuit switched environment for this archi tecture, connecting local memories to the common main memory. The invention may best be understood by reference

at the input and hardware to permit the bypass, when desired, of the extra stage or the output stage. Thus, the

to the following detailed description and accompanying

ESC has a relatively low incremental cost over the

ings:

Generalized Cube (and its equivalent networks). The 65

FIG. 1 is a block diagram of the so-called General ized n stage Cube with n=3, 2"=N= 8 (0-7) inputs and includes simplified block diagrams of the four states of an interchange box;

extra stage provides an additional path from each source to each destination. The known useful attributes

of partitionability (see H. J. Siegel, “The theory under

drawings which illustrate the invention. In the draw

3

4,523,273

4

FIG. 2 is a block diagram of an Extra Stage Cube

more independent SIMD machines, e.g., MAP (see G.

(ESC) network constructed according to the invention with N=8;

J. Nutt, “Microprocessor implementation of a parallel processor," in Proc. 4th Symp. Comput. Architecture, March 1977, pp. 147-152). A partitionable SIMD

FIG. 3a is a detail of an interchange box constructed according to the present invention with a multiplexer

/MIMD machine is a system which can be con?gured

and a demultiplexer for enabling and disabling; as one or more independent SIMD and/or MIMD ma FIG. 3b is an interchange box constructed according chines, e.g., the DCA (see S. I. Kartashev and S. P. to the present invention enabled; Kartashev, “A multicomputer system with dynamic FIG. 3c is an interchange box constructed according architecture," IEEE Trans. Comput. vol. C-28, pp. 10 704-720, October 1979) and TRAC (see U. V. Premku to the present invention disabled; FIG. 4 illustrates in heavier lines a path used when mar, R. Kapur, M. Malek, G. J. Lipovski, and P. Horne, routing from 1 to 4 in a fault-free ESC; "Design and implementation of the banyan interconnec FIG. 5 illustrates in heavier lines a broadcast path tion network in TRAC," in Proc. AFIPS 1980 Nat. used when broadcasting from 5 to 2, 3, 6, and 7 in a Comput. Conf, June 1980, pp. 643-653) supersystems. fault-free ESC; The Extra Stage Cube (ESC) network is useful in FIG. 6 illustrates a Generalized Cube network with

large-scale SIMD, MIMD, MSIMD, and partitionable

N=8 partitioned into two subnetworks of size N'=4, based on the high-order bit position, with the A and B labels denoting the two subnetworks; FIG. 7 illustrates an ESC network with N=8 parti tioned into two subnetworks of size N'=4, based on the high-order bit position, with the A and B labels denot ing the two subnetworks;

SIMD/MIMD supersystems. It can be de?ned by first considering the Generalized Cube network from which it is derived. The Generalized Cube network is a multi

stage cube-type network topology which was presented in Siegel and Smith, supra. This network has N input ports and N output ports, where N=2". It is shown in

FIG. 1 for N= 8. The network ports are numbered from 0 to N- 1. Input and output ports are network inter N=8 which is partitionable on the low-order bit posi 25 faces to external devices called sources and destinations, FIG. 8 illustrates a network of the ESC class with

tion; and FIG. 9 illustrates a network of the ESC class with

N: S which implements the cube functions in the order

cubeg, cubei, cube-2, cuben from input to output. DEFINITION OF THE ESC AND ITS RELATION TO THE GENERALIZED CUBE

An SIMD (single instruction stream-multiple data stream) (see Flynn, supra) machine typically includes a

respectively, which have addresses corresponding to their port numbers. The Generalized Cube topology has n=log2 N stages, where each stage includes a set of N lines connected to N/2 interchange boxes. Each inter change box is a two-input, two-output device and is individually controlled. An interchange box can be set to one to four legitimate states. Let the upper input and output lines be labeled i and the lower input and output lines be labeled j. The four legitimate states are: (l)

control unit, a number of processors, an equal number 35 straight-input i to output i, input 1' to output j; (2) of memory modules, and an interconnection network. exchange—input i to output j, input j to output i; (3) The control unit broadcasts instructions to all of the lower broadcast-input j to outputs i and j; and (4) processors, and all active processors execute the same upper broadcast-—input i to outputs i andj (see Lawrie, instruction at the same time. Thus, there is a single supra). This is shown in FIG. 1. instruction stream. Each active processor executes the 40 The interconnection network can be described as a instruction on data in its own associated memory mod set of interconnection functions, where each is a permu

ule. Thus, there are multiple data streams. The intercon

tation (bijection) on the set of interchange box input

nection network, sometimes referred to as an alignment or permutation network, provides a communications

/output line labels (see H. J. Siegel, “Analysis tech

facility for the processors and memory modules (see H. J. Siegel, “A model of SIMD machines and a compari son of various interconnection networks," IEEE Trans. Comput., vol. C-28, pp. 907-917, December 1979). The

Trans. Comput., vol. C-26, pp. 153-161, February 1977). When interconnection function f is applied, input

Massively Parallel Processor (MPP) (see K. E. Batcher, “Design of a massively parallel processor,” IEEE Trans. Comput., vol. C-29, pp. 836-840, September 1980) is an example of an SIMD supersystem.

An MIMD (multiple instmction stream—multiple data stream) machine (see Flynn, supra) typically com

niques for SIMD machine interconnection networks and the effects of processor address masks," IEEE S is connected to output f(S)=D for all 5, O§S

4,523,273

5

6

source and destination addresses will be the same, i.e.,

where O§i