CP400N Parallel Programming, WINTER 2012, WLU

Parallel computers, or supercomputers or high-performance clusters are ubiquitous today in Science and Engineering. Parallel programming requires inventing new algorithms and programming techniques. This course will cover the fundamental paradigms of parallel programming, with an emphasis on problem solving and actual applications. The parallel programming concepts and algorithms will be illustrated via implementations in OpenMP and MPI (Message Passing Interface), as well as serial farming.

Important Course Information

Prerequisites

CP217 (proficiency in C)

Dr. Ilias S. Kotsireas,  Office 2076A, Office Hours 24/7 and by appointment, Phone 884-0710 ext. 2218# E-Mail: ikotsireATwlu.ca

Course Textbook    

Principles of Parallel Programming Calvin Lin; Lawrence Snyder Addison-Wesley 2009 ISBN-10 0-321-48790-7 ISBN-13 978-0-321-48790-2

Textbook Support Web Site

Course Topics

1. Historical overview and evolution of parallel computing
2. Fundamental concepts of parallelism
3. Parallel computer memory architectures: shared memory, distributed memory, hybrid distributed-shared memory
4. Parallel computer system architectures: Multi-Core, SMP, Clusters, Supercomputers, Flynn's Taxonomy
5. Performance, Speedup, Scalability, Amdahl's law, Gustafson-Barsis's law, Karp-Flatt metric, Isoefficiency relation
6. Load balancing: static, dynamic, termination detection
7. Abstractions for parallel programming: Data Parallelism, Task Parallelism, Reduce and Scan, work assignment, Pipelining
8. Programming paradigms for parallel computing: OpenMP, MPI, MapReduce/Google, Cilk, Cilk++, CUDA/GPU, Implicit Parallelism (Serial Farming)
9. Applications: parallel sorting, parallel matrix-vector multiplication, parallel matrix multiplication, parallel searching, Floyd's algorithm, Sieve of Eratosthenes, Circuit Satisfiability

Class Schedule, Winter Semester Timetable

              |     Mon     |     Wed     |    
              |  2:30-3:50  |  2:30-3:50  |  
              |    BA112    |    BA112    |   
______________|_____________|_____________|__
              |             |             |             
Week 1:       |             |    Jan 04   |  
Week 2:       |  Jan 09     |    Jan 11   |  
Week 3:       |  Jan 16     |    Jan 18   |  
Week 4:       |  Jan 23     |    Jan 25   |  
Week 5:       |  Jan 30     |    Feb 01   |  
Week 6:       |  Feb 06     |    Feb 08   |  
Week 7:       |  Feb 13     |    Feb 15   |  
______________|_____________|_____________|__
              |
Reading Week  | Feb 20-24  N o  C o u r s e s
______________|______________________________
              |             |             |             
Week 8:       |  Feb 27     |   Feb 29    |  
Week 9:       |  Mar 05     |   Mar 07    |  
Week 10:      |  Mar 12     |   Mar 14    |  
Week 11:      |  Mar 19     |   Mar 21    |  
Week 12:      |  Mar 26     |   Mar 28    |  
              |  Apr 02     |             |             
______________|_____________|_____________|___
              |             |             |             

Course Requirements/Student Evaluation

• The course grade is computed based on the 5 components: A1, A2, M, TP, RAP, as follows:
  
  A1*(15/100) + A2*(15/100) + M*(30/100) + TP*(30/100) + RAP*(10/100)

• (A1) Assignment 1: 15%, release date: Monday, January 16, 2012, due date: Monday, January 30, 2012.

• (A2) Assignment 2: 15%, release date: Monday, February 6, 2012, due date: Monday, February 27, 2012.

• Important Information regarding assignment submission:
  • All assignment submissions will be by e-mail only.

  • All assignment submissions will be acknowledged by e-mail.

  • All assignment submissions must be typeset (LaTeX, Word).

  • All assignment submissions must be by .pdf file attachment only.

  • Send one .pdf file only, for the entire assignment.

  • Use the following naming schemes, for your A1 and A2 .pdf files:
    CP400N-A1-yourFirstName-yourLastName.pdf
    CP400N-A2-yourFirstName-yourLastName.pdf
    (these are dash characters, not underscores)

  • Do not use a cover page. Instead, make sure that all pages of your .pdf file are numbered and on each page include a header with your name, course code, date, and A1 (or A2)

  • Assignment submissions that violate any of the above requirements, will not be accepted/marked.

• (M) Midterm: 30%, Wednesday, February 15, 2012, in class.

• (TP) Term Project: 30%, due date: Monday, April 2, 2012.

  All students will be required to prepare a Term Project, details in class. Students will form groups (of no more than 4 students each) to work on the Term Project collaboratively. Each group will have to deliver a project document and arrange for a project demonstration.

• (RAP) Research Article Presentation: 10%, Week 12 of classes.

  Each student is required to study, understand and present a research article (published in a research journal or a research conference) relevant to parallel programming research. Half-hour presentations (20-min talk + questions/answers) will be scheduled for the last week of courses. Students are required to work on this project individually.
  Some sources where you can find such research papers are: (on-line access also from Trellis)

  1. International Journal of Parallel Programming (Springer)
  2. Parallel Computing Systems & Applications (Elsevier)
  3. Journal of Parallel and Distributed Computing (Elsevier)
  4. Encyclopedia of Parallel Computing