CP331 Parallel Programming, WINTER 2016, WLU

Parallel computers, or supercomputers or high-performance clusters are ubiquitous today in Science and Engineering. Parallel programming requires inventing new algorithms and programming techniques. This course will cover the fundamental paradigms of parallel programming, with an emphasis on problem solving and actual applications. The parallel programming concepts and algorithms will be illustrated via implementations in OpenMP and MPI (Message Passing Interface), as well as serial farming.

Important Course Information

Prerequisites

CP217 (proficiency in C)

Dr. Ilias S. Kotsireas,  Office 2076A, Office Hours 24/7 and by appointment, Phone 884-0710 ext. 2218# E-Mail: ikotsireATwlu.ca

Course Topics

1. Historical overview and evolution of parallel computing
2. Fundamental concepts of parallelism
3. Parallel computer memory architectures: shared memory, distributed memory, hybrid distributed-shared memory
4. Parallel computer system architectures: Multi-Core, SMP, Clusters, Supercomputers, Flynn's Taxonomy
5. Performance, Speedup, Scalability, Amdahl's law, Gustafson-Barsis's law, Karp-Flatt metric, Isoefficiency relation
6. Load balancing: static, dynamic, termination detection
7. Abstractions for parallel programming: Data Parallelism, Task Parallelism, Reduce and Scan, work assignment, Pipelining
8. Programming paradigms for parallel computing: OpenMP, MPI, MapReduce/Google, Cilk, Cilk++, CUDA/GPU, Implicit Parallelism (Serial Farming)
9. Applications: parallel sorting, parallel matrix-vector multiplication, parallel matrix multiplication, parallel searching, Floyd's algorithm, Sieve of Eratosthenes, Circuit Satisfiability

Class Schedule, Winter Semester Timetable

              |     Tue     |     Thu     |    
              |  5:30-7:00  |  5:30-7:00  |  
              |    N1057    |    N1057    |   
______________|_____________|_____________|__
              |             |             |             
Week 1:       |   Jan 05    |   Jan 07    |
Week 2:       |   Jan 12    |   Jan 14    |
Week 3:       |   Jan 19    |   Jan 21    |
Week 4:       |   Jan 26    |   Jan 28    |
Week 5:       |   Feb 02    |   Feb 04    | 
Week 6:       |   Feb 09    |   Feb 11    |
______________|_____________|_____________|__
              |
Reading Week  | Feb 15-18  N o  C o u r s e s
______________|______________________________
              |             |             |
Week 7:       |  Feb 23     |   Feb 25    |
Week 8:       |  Mar 01     |   Mar 03    |
Week 9:       |  Mar 08     |   Mar 10    |
Week 10:      |  Mar 15     |   Mar 17    |
Week 11:      |  Mar 22     |   Mar 24    |
Week 12:      |  Mar 29     |   Mar 31    |
______________|_____________|_____________|___
              |             |             |             

Course Requirements/Student Evaluation

• The course grade is computed based on the 5 components: A1, A2, M, TP, RAP, as follows:
  
  A1*(15/100) + A2*(15/100) + M*(30/100) + TP*(30/100) + RAP*(10/100)

• (A1) Assignment 1: 15%, release date: TBA, due date: TBA.

• (A2) Assignment 2: 15%, release date: TBA, due date: TBA.

• Important Information regarding assignment submission:
  • Students will form groups (of 3 students each) to work on the assignments collaboratively.

  • Each group of 3 students will receive a group number, on a first-come-first-served basis.

  • One of the three students will be designated responsible for assignment submissions.

  • All assignment submissions will be by e-mail only.

  • All assignment submissions will be acknowledged by e-mail.

  • All assignment submissions must be typeset (LaTeX, Word).

  • All assignment submissions must be by .pdf file attachment only.

  • Send one .pdf file only, for the entire assignment.

  • Use the following naming schemes, for your A1 and A2 .pdf files:
    CP331-A1-yourGroupNumber-1stLastName-2ndLastName-3rdLastName.pdf
    CP331-A2-yourGroupNumber-1stLastName-2ndLastName-3rdLastName.pdf
    (these are dash characters, not underscores)

  • Do not use a cover page. Instead, make sure that all pages of your .pdf file are numbered and on each page include a header with your name, course code, date, and A1 (or A2)

  • Assignment submissions that violate any of the above requirements, will not be accepted/marked.

• (M) Midterm: 30%, Thursday February 11, 2016 (in class).

• (TP) Term Project: 30%, due date: Monday April 4, 2016.

  All students will be required to prepare a Term Project, details in class. Students will form groups (of 3 students each) to work on the Term Project collaboratively. Each group will have to deliver a project document and arrange for a project demonstration.

• (RAP) Research Article Presentation: 10%, Week 12 of classes.

  Each student is required to study, understand and present a research article (published in a research journal or a research conference) relevant to parallel programming research. Each student is required to e-mail the course instructor a .pdf copy of his/her proposed RAP article, using the following naming scheme: CP331-RAP-yourFirstName-yourLastName.pdf . RAPs (10-min talk + questions/answers) will be scheduled for the last week of courses. Students are required to work on this project individually.
  Some sources where you can find such research papers are: (on-line access also from Trellis)

  1. International Journal of Parallel Programming (Springer)
  2. Parallel Computing Systems & Applications (Elsevier)
  3. Journal of Parallel and Distributed Computing (Elsevier)
  4. Encyclopedia of Parallel Computing