CP400N Parallel Programming, WINTER 2013, WLU

Parallel computers, or supercomputers or high-performance clusters are ubiquitous today in Science and Engineering. Parallel programming requires inventing new algorithms and programming techniques. This course will cover the fundamental paradigms of parallel programming, with an emphasis on problem solving and actual applications. The parallel programming concepts and algorithms will be illustrated via implementations in OpenMP and MPI (Message Passing Interface), as well as serial farming.

Important Course Information

Prerequisites

CP217 (proficiency in C)

Dr. Ilias S. Kotsireas,  Office 2076A, Office Hours 24/7 and by appointment, Phone 884-0710 ext. 2218# E-Mail: ikotsireATwlu.ca

Course Topics

1. Historical overview and evolution of parallel computing
2. Fundamental concepts of parallelism
3. Parallel computer memory architectures: shared memory, distributed memory, hybrid distributed-shared memory
4. Parallel computer system architectures: Multi-Core, SMP, Clusters, Supercomputers, Flynn's Taxonomy
5. Performance, Speedup, Scalability, Amdahl's law, Gustafson-Barsis's law, Karp-Flatt metric, Isoefficiency relation
6. Load balancing: static, dynamic, termination detection
7. Abstractions for parallel programming: Data Parallelism, Task Parallelism, Reduce and Scan, work assignment, Pipelining
8. Programming paradigms for parallel computing: OpenMP, MPI, MapReduce/Google, Cilk, Cilk++, CUDA/GPU, Implicit Parallelism (Serial Farming)
9. Applications: parallel sorting, parallel matrix-vector multiplication, parallel matrix multiplication, parallel searching, Floyd's algorithm, Sieve of Eratosthenes, Circuit Satisfiability

Class Schedule, Winter Semester Timetable

              |     Mon     |     Wed     |    
              |  2:30-3:50  |  2:30-3:50  |  
              |    N1042    |    N1042    |   
______________|_____________|_____________|__
              |             |             |             
Week 1:       |  Jan 07     |    Jan 09   |  
Week 2:       |  Jan 14     |    Jan 16   |  
Week 3:       |  Jan 21     |    Jan 23   |  
Week 4:       |  Jan 28     |    Jan 30   |  
Week 5:       |  Feb 04     |    Feb 06   |  
Week 6:       |  Feb 11     |    Feb 13   |  
______________|_____________|_____________|__
              |
Reading Week  | Feb 18-22  N o  C o u r s e s
______________|______________________________
              |             |             |
Week 7:       |  Feb 25     |   Feb 27    |
Week 8:       |  Mar 04     |   Mar 06    |  
Week 9:       |  Mar 11     |   Mar 13    |  
Week 10:      |  Mar 18     |   Mar 20    |  
Week 11:      |  Mar 25     |   Mar 27    |  
Week 12:      |  Apr 01     |   Apr 03    |  
______________|_____________|_____________|___
              |             |             |             

Course Requirements/Student Evaluation

• The course grade is computed based on the 5 components: A1, A2, M, TP, RAP, as follows:
  
  A1*(15/100) + A2*(15/100) + M*(30/100) + TP*(30/100) + RAP*(10/100)

• (A1) Assignment 1: 15%, release date: Monday, January 21, 2013, due date: Monday, February 04, 2013.

• (A2) Assignment 2: 15%, release date: Monday, February 11, 2013, due date: Monday, February 25, 2013.

• Important Information regarding assignment submission:
  • All assignment submissions will be by e-mail only.

  • All assignment submissions will be acknowledged by e-mail.

  • All assignment submissions must be typeset (LaTeX, Word).

  • All assignment submissions must be by .pdf file attachment only.

  • Send one .pdf file only, for the entire assignment.

  • Use the following naming schemes, for your A1 and A2 .pdf files:
    CP400N-A1-yourFirstName-yourLastName.pdf
    CP400N-A2-yourFirstName-yourLastName.pdf
    (these are dash characters, not underscores)

  • Do not use a cover page. Instead, make sure that all pages of your .pdf file are numbered and on each page include a header with your name, course code, date, and A1 (or A2)

  • Assignment submissions that violate any of the above requirements, will not be accepted/marked.

• (M) Midterm: 30%, Wednesday, February 27, 2013, in class.

• (TP) Term Project: 30%, due date: Wednesday, April 3, 2013.

  All students will be required to prepare a Term Project, details in class. Students will form groups (of no more than 4 students each) to work on the Term Project collaboratively. Each group will have to deliver a project document and arrange for a project demonstration.

• (RAP) Research Article Presentation: 10%, Week 12 of classes.

  Each student is required to study, understand and present a research article (published in a research journal or a research conference) relevant to parallel programming research. Half-hour presentations (20-min talk + questions/answers) will be scheduled for the last week of courses. Students are required to work on this project individually.
  Some sources where you can find such research papers are: (on-line access also from Trellis)

  1. International Journal of Parallel Programming (Springer)
  2. Parallel Computing Systems & Applications (Elsevier)
  3. Journal of Parallel and Distributed Computing (Elsevier)
  4. Encyclopedia of Parallel Computing