ÍÃ×ÓÏÈÉú

Catalog description:

Introduction to the practical use of multi-processor workstations and supercomputing clusters. Developing and using parallel programs for solving computationally intensive problems. The course builds on basic concepts of programming and problem solving.

Prerequisite:

CSE 381

Required topics (approximate weeks allocated):

• Introduction to parallel programming and high performance distributed computing (HPDC) (1)
  
  • Motivation for HPDC
  • Review of parallel programs and platforms
  • Implicit parallelism and limitations of instruction level parallelism (ILP)
  • Survey of architecture of commonly used HPDC platforms
• Concurrency and parallelism (1)
  • Introduction to concurrency & Parallelism
  • Levels of parallelism
  • Instruction level parallelism
  • SIMD versus MIMD
• Review of C programming language and the Linux environment (1.5)
  
  • Review of basic programming constructs
  • Applying Java/C++ syntax and semantics to C language
  • Introduction to problem solving using the C language
  • Introduction to Linux
  • C programming using Linux
  • C structures
• Exploring instruction level parallelism (1)
  • Review of instruction level parallelism and sources of hazards
  • Concepts of hazard elimination via code restructuring (dependency reduction, loop unrolling)
  • Timing and statistical comparison of performance of c programs
• Introduction to parallel programming (2)
  • Principles of parallel algorithms
  • Effects of synchronization and communication latencies
  • Overview of physical and logical communication topologies
  • Using MPE for parallel graphical visualization (parallel libraries)
• Introduction to message passing paradigm (.5)
  • Principles of message-passing programming
  • The building blocks of message passing
• Programming in MPI (3)
  • Introduction to MPI: The Message Passing Interface
  • MPI Fundamentals
  • Partitioning data versus partitioning control
  • Blocking communications and parallelism
  • MPI communication models
  • Blocking vs. non-blocking communication and impacts of parallelism
  • Developing MPI programs that exchange derived data types
  • Create MPI programs that use structure derived data types
  • Review of portability and interoperability issue
• Performance profiling (1)
  • Using software tools for performance profiling
  • Performance profiling of MPI programs
  • Speedup anomalies in parallel algorithms
• Collective communications (2)
  
  • Introduction to collective communications
  • Distributed debugging
  • Introduction to MPI scatter/gather operations
  • Exploring the complete collective communication operations in MPI
• Scalability and performance (1)
  • Understanding notions of scalability and performance
  • Metrics of scalability and performance
  • Asymptotic analysis of scalability and performance
• Exams/Reviews (1)

Learning Outcomes:

• Identify various forms of parallelism, their application, advantages, and drawbacks
• Effectively utilize instruction level parallelism
• Effectively utilize multi-core CPUs and multithreading
• Analyze and implement parallel and distributed programs using the Message Passing Interface (MPI) that solves a stated problem in a clean, robust, efficient, and scalable manner
• Describe and empirically demonstrate concepts of parallel efficiency and scalability
• Apply concepts to solve computationally demanding problems
• Analyze experimental results and draw inferences to verify hypotheses

Graduate students:

Students taking the course for graduate credit will be expect​ed to apply ​course ​concepts to solve computationally demanding problems​, analyze experimental results​,​ and draw inferences to verify hypotheses.