RIT Computer Engineering


Sunday, 25-Aug-2019 07:10:53 EDT

40 Visitors Since 10-March-2003


  Assignment #1, Due Tuesday, April 8:    PDF,   Word97

  Assignment #2, Due Thursday, April 24:  PDF,   Word97

Files for question 1:
PVM Master Program: psum.c PVM Slave Program:spsum.c Numbers file: rand_data.txt

  Assignment #3, Due Tuesday, May 6:  PDF,   Word97
Files for question 1:
problem1-region-color.bmp problem1-geometry.txt problem1-colormap.txt


For the following lecture notes you can download or view a lecture as an Acrobat PDF file, or as a Microsoft Powerpoint97 file:

3-11-2K3 The Need and Feasibility of Parallel Computing, Technology Trends, Microprocessor Performance Attributes, Goal of Parallel Computing. Computing Elements, Programming Models, Flynn's Classification, Multiprocessors Vs. Multicomputers. Current Trends In Parallel Architectures, Communication Architecture.
(PCA Chapter 1.1, 1.2)

3-13-2K3 Parallel Architectures Convergence: Naming, Operations, Ordering, Replication. Communication Cost with respect to various programming models, Communication Cost Model.
(PCA Chapter 1.2, 1.3)

3-18-2K3 Parallel Programs: Conditions of Parallelism. Asymptotic Notations for Algorithm Analysis, PRAM. Levels of Parallelism, Hardware Vs. Software Concurrency. Amdahl’s Law, DOP, Concurrency Profile. Steps in Creating Parallel Programs: Decomposition, Assignment, Orchestration, Mapping.
(PCA Chapter 2.1, 2.2)

3-20-2K3 Parallelization of An Example Program: Ocean simulation Iterative equation solver.
(PCA Chapter 2.3)

3-27-2K3 Considerations in Parallel Program Creation Steps for Performance.
(PCA Chapter 3)

4-1-2K3 Cluster Computing: Origins, Broad Issues in Heterogeneous Computing (HC). Message-Passing Programming. Overview of Parallel Virtual Machine (PVM).
(PP Chapter 2, Appendix A, PVM & HC References Below)

4-8-2K3 Overview of Message Passing Interface (MPI 1.2).
(PP Chapter 2, Appendix B, MPI References Below)

4-15-2K3 Parallel Programming Techniques and Examples. Massively Parallel Computations: Pixel-based Image Processing. Divide-and-conquer Problem Partitioning: Parallel Bucket Sort, Numerical Integration, Gravitational N-Body Problem. Pipelined Computations: Addition, Insertion Sort, Solving Upper-triangular System of Linear Equations. Synchronous Iteration: Barriers, Iterative Solution of Linear Equations. Dynamic Load Balancing: Centralized, Distributed, Moore's Shortest Path Algorithm.
(PP Chapters 3-7, 11)

4-17-2K3 Network Properties and Requirements For Parallel Computing. Static Point-to-point Connection Network Topologies. Network Embeddings. Dynamic Connection Networks.
(PCA Chapter 10, handout)

4-22-2K3 Shared Memory Multiprocessors. The Cache Coherence Problem. Memory Access Consistency Models. Cache Coherence Approaches. Snoopy Bus Protocols: Write-invalidate: MSI, MESI, Write-Update: Dragon.
(PCA Chapter 5, 9.1, handout)

4-29-2K3 Parallel System Performance: Evaluation & Scalability. Workload Selection. Parallel Performance Metrics Revisited. Application Models of Parallel Computers. Parallel System Scalability.
(PP chapter 1, PCA Chapter 4, handout)

5-6-2K3 Scalable Distributed Memory Machines Design Choices. MPPs Hardware Scalability Issues: Node Integration Level, Network, Communication Assist, Cost. MPP Physical Scaling Examples: nCUBE/2, CM-5, IBM SP-2, iWARP, Intel Paragon. Extent of CA Interpretation (Functionality of CA) & Examples: Blind Physical DMA (nCUBE/2), System-Level Vs. User-Level Ports (CM-5, iWARP), Global Address Space (Cray T3E).
(PCA Chapter 7)

5-13-2K3 Cache Coherence in Scalable Distributed Memory Machines: Hierarchical Snooping, Directory-based cache coherence.
(PCA Chapter 8)


Tuesday and Thursday: 4:00-5:50 PM, 17/1545


Dr. Muhammad Shaaban
e-mail: meseec@rit.edu
Office: 17-2507 X2373

Office Hours:
My Spring 2003 schedule


Current:    http://www.rit.edu/~meseec/eecc756-spring2003/
Spring 2002: http://www.rit.edu/~meseec/eecc756-spring2002/
Spring 2001: http://www.rit.edu/~meseec/eecc756-spring2001/
Spring 2000: http://www.rit.edu/~meseec/eecc756-spring2000/
Spring 99: http://www.rit.edu/~meseec/eecc756-spring99/
Spring 98: http://www.rit.edu/~meseec/eecc756-spring98/


This course covers a number of issues involved in the design and utilization of high performance parallel computing systems. This includes: parallel computer models, the concept of scalable performance, the memory hierarchy, cache coherence issues, parallel and scalable architectures, parallel programming concepts. A number of current parallel machines will be studied.


Computer Architecture EECC-551.



Parallel Computer Architecture (PCA): A Hardware/Software Approach, David E. Culler, Jaswinder P. Singh, Morgan Kaufmann Publishers, 1999.

Parallel Programming (PP): Techniques and Applications Using Networked Workstations and Parallel Computers, Barry Wilkinson, Micheal Allen, Prentice Hall, 1998.  ISBN: 0-13-671710-1  Buy online from Amazon.com


Designing and Building Parallel Programs, Ian Foster, Addison-Wesley, 1995, complete textbook online, ( includes a chapter on MPI).

Scalable Parallel Computing, Kai Hwang, Zhiwei, McGraw-Hill, 1998.

Advanced Computer Architecture: Parallelism, Scalability, Programmability, Kai Hwang, McGraw-Hill, 1993.

Parallel Virtual Machine (PVM):

PVM (Parallel Virtual Machine) Home Page

PVM: Parallel Virtual Machine: A Users' Guide and Tutorial for Networked Parallel Computing, Al Geist(Editor), et al, MIT Press, 1994, complete online version , PDF version , postscript version.

Advanced Tutorial on PVM 3.4 New Features and CapabilitiesAl Geist, Presented at EuroPVM-MPI'97, 1997.

Message-Passing Interface (MPI):

MPI Home Page.

MPICH Home Page A Portable Implementation of MPI.

Beginner's Guide to MPI , University of Delaware.

MPI: The Complete Reference , Marc Snir et al., First Edition, 1995 (html version of book).

Heterogeneous Computing (HC)

Heterogeneous Computing: Challenges and Opportunities, PDF,
Ashfaq A. Khokhar, Viktor K. Prasanna, Muhammad E. Shaaban, Cho-Li Wang
IEEE Computer, June 1993 (Vol. 26, No. 6), pp. 18-27.

Heterogeneous Distributed Computing , PDF,
Muthucumaru Maheswaran, Tracy D. Braun, Howard Jay Siegel,
Edited version of a chapter appearing in the Encyclopedia of Electrical and Electronics Engineering, J. G. Webster, editor, John Wiley & Sons, New York, NY, 1999 Vol. 8, pp. 679-690.

A Comparison Study of Static Mapping Heuristics for a Class of Meta-tasks on Heterogeneous Computing Systems, PDF,
Tracy D. Braun, Howard Jay Siegel, Noah Beck, Ladislau L. Boloni, Albert I. Reuther, Mitchell D. Theys, Bin Yao, Richard F. Freund
Proceedings 8th Heterogeneous Computing Workshop, 1999. (HCW 1999), 1999, pp. 15-29.

Segmented min-min: a static mapping algorithm for meta-tasks on heterogeneous computing systems, PDF,
Min-You Wu, Wei Shu, H. Zhang,
Proceedings. 9th Heterogeneous Computing Workshop, 2000. (HCW 2000), 2000, pp. 375 -385.

Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing, PDF,
H. Topcuoglu, S. Hariri, M.Y. Wu,
IEEE Transactions on Parallel and Distributed Systems, March 2002 (Vol. 13, No. 3).

Greedy Heuristics for Resource Allocation in Dynamic Distributed Real-Time Heterogeneous Computing Systems, PDF,
S. Ali, J. Kim, H. J. Siegel, A. Maciejewski, Y. Yu, S. Gundala, S. Gertphol, V. K. Prasanna,
Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA '02), June 2002, (Volume 2), pp. 519-530.

Selected papers.

EECC756 Spring 1999 Class Projects.


Homework assignments: 30%
Exam: 40%
Class project: 30%


Attending all lecture sessions is expected.


Week1: Motivation for Parallel Computing, Parallel Programming Models, Parallel Architectures. Communication Cost Model
Week2: Parallel Program characteristics and creation Steps.
Program Parallelization Example: Ocean simulation Iterative equation solver.
Week3: Parallel Programming for Performance.
Week4: Heterogeneous & Cluster Computing. Message-Passing Programming & Environments.
Parallel Virtual Machine (PVM). Message Passing Interface (MPI).
Week5: Message-Passing Computing Examples.
Week6: Network Requirements For Parallel Computing. Static Point-to-point Connection Network Topologies. Dynamic Connection Networks.
Week7: Parallel System Performance: Evaluation & Scalability Metrics.
Week8: Shared Memory Multiprocessors. The Cache Coherence Problem. Scalable Distributed Memory Parallel Machines.
Week9: Cache Coherence in Scalable Distributed Memory Parallel Machines.
Week10: Exam/Project Presentations.

  This page is 27Kbytes long and was last modified on:   Monday, 25-Feb-2008 17:23:18 EST.

Made with at least 30% post-consumer recycled bits