Inf-2202 Concurrent and Data-Intensive Programming

Administrative information

Administrative course information is available here

We use the inf-2202-f15@list.uit.no mailing list to send important information.

We have the following rooms and hours:

Staff

Lecture plan

Lecture Date Subject Lecturer
L1 Fri 14.08 Introduction Lars Ailo
L2 Thu 20.08 Threads and synchronization primitives Lars Ailo
L3 Thu 27.08 Guest lecture: Go Giacomo Tartari
L4 Thu 03.09 Parallel architectures Lars Ailo
L5 Thu 10.09 Parallel programs Lars Ailo
L6 Tue 15.09 Programming for performance Lars Ailo
L7 Thu 24.09 Performance evaluation Lars Ailo
L8 Thu 01.10 Parallel program performance evaluation Lars Ailo
  Thu 08.10 Postponed due to a major water leak! -
L10 Thu 15.10 Guest lecture: Scala and Spark Inge Alexander Raknes
L9 Thu 22.10 Data-intensive computing Lars Ailo
L11 Thu 29.10 Spark libraries Lars Ailo
  Thu 05.11 Cancelled Lars Ailo
L12 Tue 10.11 Guest lecture: Stallo (no slides) Steinar Trædal-Henden
L13 Tue 17.11 Summary (no slides) Lars Ailo
  Thu 26.11 Exam -

Readings

All lecture noets are Mandatory, and in addition unless otherwise noted:

  1. Introduction
    • None
  2. Threads and synchronization primitives (operating systems course recap):
    • Modern operating systems, 3ed, Andrew S. Tanenbaum. Prentice Hall. 2007. Chapters: 2.2, 2.3, 2.5, 10.3, 11.4
    • Alternative to MOS: another operating systems textbook: the chapters about threading, IPC mechanisms, and classical IPC problems.
  3. Go
  4. Parallel architectures
    • Computer Organization and Design: the Hardware/Software Interface, 4ed. David A. Patterson, John L. Hennessy. Morgan Kaufmann. 2011. Chapter 8: “Multicores, Multiprocessors, and Clusters”.
  5. Parallel programs
    • None
  6. Programming for performance
    • None
  7. Performance evaluation
    • None
  8. Parallel performance evaluation
  9. Data-intensive computing
  10. Scala and Spark
    • None
  11. Spark libraries
    • Optional: videos, slides, and research papers at: http://spark.apache.org/documentation.html

The following are suggested additional readings:

Mandatory assignments

Project Start Due Subject Lecturer
P1 18.08.15 14.09.15 Concurrent B+ trees (private repository and public zip) Ibrahim
P2 17.09.15 12.10.15 Deduplication (private repository and public zip) Ibrahim
P3 13.10.15 06.11.15 PageRank using Spark on AWS (private repository and public zip ) Lars Ailo and Ibrahim

Note! The mandatory assignment text and pre-code are available in private repositories accessible only for the members of the uit-inf-2202 github organization.

Exercises

  1. Introduction
    1. None
  2. Threads and synchronization primitives
    1. Compare the overhead of forking a process vs. creating a Pthread
    2. Compare the overhead of forking a process vs. creating a Python thread
    3. Implement a solution the following classical IPC problems using pthreads/Python threads and semaphores/condition variables. Note that you also need to generate a use case, test data, and useful output:
      1. Producer/ consumer
      2. Reader/ writer
      3. Sleeping barber
      4. Dining philosophers
    4. Modify the code in 3) to use message passing.
  3. Go
    1. Take a tour of Go
    2. Implement the classical IPC problems in exercise 2.3. in Go.
  4. Parallel architectures
    1. None
  5. Parallel programs
    1. Implement a simpliefied BLAST search program in Go that does similarity search on two lists of random DNA sequences.
    2. Implement a heat distribution program using Pthreads or (/and) Go.
  6. Programming for performance
    1. Implement a tuple space in Python with semantics similar to Linda. Use your tuple space to implement a parallel version of Mandelbrot that uses dynamic assignment (pool of tasks).
  7. Performance evaluation
    • None
  8. Parallel program performance evaluation
    • Go through either the debunking or ninja paper and find out how they did each of the “Steps for a performance evaluation study”.
  9. Data-intensive computing
    • Create an account at AWS and calculate the approximate cost for analyzing 1TB and 1PB of data.
    • Implement word count in MapReduce and run it on uvrocks or AWS.
    • Implement grep in MapReduce and run it on uvrocks or AWS.
  10. Scala and Spark
    • Run the provided WordCount in assignment 3 on AWS
    • Implement grep in Scala and run it on AWS
  11. Spark libraries
    • Refactor your assignment 3 code to use GraphX
  12. Stallo
    • None
  13. Summary lecture