Master Program (Laurea Magistrale) in Computer Science and Networking Academic Year 2011-2012

High Performance Computing Systems and Enabling Platforms Marco Vanneschi Department of Computer Science, University of Pisa

Master Program (Laurea Magistrale) in Computer Science and Networking

High Performance Computing Systems and Enabling Platforms Marco Vanneschi

Course Introduction www.di.unipi.it/vannesch section: Teaching

SPA: course objectives • Providing a solid framework of concepts and techniques in high-performance computing – Parallelization methodology and models – Support to parallel programming models and software development tools – Parallel Architectures – Performance evaluation (cost models)

• Methodology for studying existing and future systems • Technology: state-of-the-art and trends – Parallel processors

– Multiprocessors – Multicore / manycore / … / GPU – Shared vs distributed memory architectures – Programming models and their support MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

3

SPA: first course in HPC and enabling platforms • Distributed Systems: Paradigms and Models (Prof. Danelutto), mandatory, 2nd semester – assumes SPA as prerequisite • Other courses in this area – Programming tools for parallel and distributed systems – Complements of distributed enabling platforms – Parallel and distributed algorithms – …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

4

Course approach: a Computer Science approach methodological knowledge

technological knowledge

• Computer Science approach o Parallel computing: computational and programming models o Cost models of parallel computations o Computing architecture has its own concepts, principles, models, and techniques o Conceptual framework in common with the other disciplines of Computer Science: •

Programming languages, algorithms, computability and complexity, …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

5

Basic background and prerequisites • An undergraduate-level course on structured computer architecture – Firmware level structuring – Assembler level, CPU architecture, compiling – Memory hierarchies and caching – Interrupt handling, exception handling – Process level, addressing space, low level scheduling, interprocess communication – Input/Output processing

• In Pisa: the course “Computer Architecture” (Undergraduate Program in Computer Science) adopts such an approach

• Precourse on Computer Architecture fundamentals MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

6

Precourse: subjects and approach 1. System structuring: level, modules, cooperation models

2. Firmware level fundamentals 3. Assembler level 4. Memory hierachies and caching

5. Process level fundamentals and run-time support • Approach of Precourse: – Main concepts and techniques on these subjects – Exercises: proposed to students and discussed (class-level or individually)

• Recommendation: Precourse-level knowledge is of fundamental importance for SPA course and exam. MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

7

Precourse: teaching material • Slides: used as course notes for the precourse – On my personal page

– Part 5 of Precourse (Process Level and Communication) is present in SPA Course Notes (Part 1, Section 3)

• Reference books: – D.A. Patterson, J.H. Hennessy, “Computer Organization and Design: the Hardware/Software Interface”, Morgan Kaufman Publishers Inc. – A. Tanenbaum, “Structured Computer Organization”, Prentice-Hall. – in Italian: M. Vanneschi, “Architettura degli Elaboratori”, PLUS, Pisa.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

8

Working approach: Precourse and SPA • As in any other course, it is fundamental to acquire skills and capabilities in concepts and principles, besides knowing the technologies. • Critical aptitude must be properly developed.

• Interaction with the teacher is strongly encouraged – Questions during the lectures – Presentation and discussion of exercises

– Question time (“orario di ricevimento”) (in Italian for Italians) • … (to be fixed) • and/or by appointment in case of collision with other courses, or other reasons. MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

9

SPA: Course Program PART 1 (about 3 CFU) 1.

Methodology for Parallel Application Structuring and Design • Parallel computation models • Parallelization methodology and techniques • Parallel paradigms • Programming models • Cost models • Static and dynamic optimizations

PART 2 (about 3 CFU) 2.

Parallel Architectures • Shared memory architectures : SMP, NUMA • Distributed memory architectures: Clusters, MPP • Interconnection networks, • Support to concurrency mechanisms • Cost models • Parallel application benchmarks • Multicore, multithreading, advanced technologies

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

10

SPA: Teaching Material • Course notes – available on my personal page – version 2011-12

• Reference books: – Patterson: see Precourse references – D.E. Culler, J.P. Singh, A. Gupta, “Parallel Computer Architecture: a Hardware/Software Approach”, Morgan Kaufman Publishers Inc. – Papers and reports

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

11

SPA: exam modality • Written + oral exam (in English or in Italian)

• Intermediate tests (midterms) ? • Exam approach: the student must be able to demonstrate a deep understanding of concepts and techniques and of their interrelationships, as well as to write / to present the answer in a clear and rigorous way. • Registration to the exam on the Official Site of Corso di Laurea: http://compass2.di.unipi.it/didattica, section Laurea Magistrale in Informatica e Netwoking, subsection “calendar”

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

12

Architetture Parallele e Distribuite (ASE) Esame di ASE (9 CFU), laurea specialistica vecchio ordinamento:

• SPA (6 CFU) • + integrazione 3 CFU (Libro PLUS, Parte IV) – implementazione di programmi paralleli in LC secondo forme di parallelismo note – architettura di CPU pipeline e valutazione di programmi

• Modalità di esame: scritto.

• All’atto dell’iscrizione: indicare ASE

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

13

Good Luck !

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

14