High Performance Computing

Instructor: Salvatore Orlando

Goals

The objective is to acquire knowledge on models of parallel architectures, paradigm and environments of parallel programming, and performance evaluation of parallel systems. The course presents the fundamental algorithmic techniques usually employed to solve in parallel significant application problems. The course is mainly focused on the use of clusters of workstations as parallel platforms.

Requirements

The students is expected to have a good background in computer architectures, operating systems, computer networks, programming.

Contents

  1. Introduction: Motivations and use cases.
  2. Parallel architectures
  3. Parallel programming paradigms
  4. Parallelization techniques and issues: Decompositions, Mapping, Load balancing.
  5. Metrics for performance evaluation.
  6. Basics of parallel algorithms.
  7. Recurrent paradigms for parallelization.
  8. Libraries and languages for parallel programming.
  9. Laboratory work: use of message passing (MPI) and multithread libraries.

Recommended Reading List

Lecture notes.
A. Grama, A. Gupta, G. Karypis, V. Kumar. Introduction to Parallel Computing, Addison-Wesley, 2003.

The Sourcebook of Parallel Computing. Edited by Jack Dongarra, Ian Foster, Geoffrey Fox, William Gropp, Ken Kennedy, Linda Torczon, Andy White, October 2002, 760 pages, ISBN 1-55860-871-0, Morgan Kaufmann Publishers.

I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, Versione online disponibile presso http://www-unix.mcs.anl.gov/dbpp.

B. Wilkinson, M. Allen. Parallel Programming: Techniques and Applications Using Networked Workstation and Parallel Computers. Prentice-Hall, 1999.

On-line Documents

Assessments

Written exam, project discussion, talk on a scholarly paper.

Written Exams: Examples of questions

  1. In the coherence protocol MESI (Modified-Exclusive-Shared-Invalidate), what is the meaning of the various states that can be associated with a cache line?
  2. SIMD and MIMD are the two main models of a parallel architecture. Discuss and exemplify.
  3. Discuss the Amdhal law for performance evaluation

Teaching Methods

Class lectures. Practical laboratory lectures. Group work for laboratory projects.

Mailing list

Students must register in moodle.unive.it by following the path:
OFFERTA FORMATIVA > Corsi di Laurea Magistrale > Dipartimento di Scienze Ambientali, Informatica e Statistica >
Informatica - Computer Science [CM9] > High Performance Computing [CM0227]

Lecture Notes

  1. Introduction
  2. Parallel Architectures (PPTX file)
  3. Message Passing Programming
  4. Shared Memory Programming
  5. Loop parallelization, SSE SIMD Programming
  6. Modeling and Performance Evaluation
  7. Parallel Programming Patterns
  8. OpenMP
  9. GP-GPU

Project A.Y. 2016/17