Parallel Programming with OpenMP Part #1

Parallel Programming with OpenMP Part #1 CS 475 Dr. Ziad Al-Sharif 1

A Programmer s View of OpenMP What is OpenMP? Open specification for Multi-Processing Standard API for defining multi-threaded shared-memory programs openmp.org Talks, examples, forums, etc. OpenMP is a portable, threaded, shared-memory programming specification with light syntax Exact behavior depends on OpenMP implementation! Requires compiler support (C/C++ or Fortran) OpenMP will: Allow a programmer to separate a program into serial regions and parallel regions, rather than T concurrently-executing threads. Hide stack management Provide synchronization constructs (syntax) OpenMP will not: Parallelize automatically ?!!! Guarantee speedup ?!!! Provide freedom from data races ?!!! 2

OpenMP Programming Model OpenMP is designed for multi-processor/multi- core/multi-processing, shared memory machines. The underlying architecture can be shared memory UMA or NUMA. Uniform Memory Access (UMA) Non-Uniform Memory Access (NUMA) Because OpenMP is designed for shared memory parallel programming: It is largely limited to single node parallelism. Typically, the number of processing elements (cores) on a node determine how much parallelism can be implemented.

OpenMP is a Thread Based Parallelism: OpenMP programs accomplish parallelism exclusively through the use of threads. A thread of execution is the smallest unit of processing that can be scheduled (LWP) by an operating system. The idea of a subroutine that can be scheduled to run autonomously might help explain what a thread is. Threads exist within the resources of a single process. Without the process, they cease to exist. Typically, the number of threads match the number of machine processors/cores (but not limited ). However, the actual use of threads is up to the application/programmer. 4

OpenMP Core Syntax Most of the constructs in OpenMP are compiler directives. Syntax: #pragma omp construct [clause [clause] ] Examples: #pragma omp parallel #pragma omp parallel num_threads(4) Function prototypes and types in the file: #include <omp.h> Most OpenMP constructs apply to a structured block . Structured block: a block of one or more statements with one point of entry at the top and one point of exit at the bottom. No premature exits from the loop allowed (for now) i.e. No break, return, exit, goto statements In general, don t jump outside of any #pragma block

Exercise 1: Hello world Write a program that prints hello world . Verify that your environment works ? gcc, WSL, or VM Write a program that prints hello world in C; no threads Verify that your C and GCC environment works ? Tryout OpenMP Write a multithreaded program that prints hello world Use / tryout the -fopenmp with the gcc compiler #include <stdio.h> #include <omp.h> int main() { // Do this part in parallel printf( "Hello, World!\n" ); return 0; } $ gcc sample.c -fopenmp OpenMP include file (Header file) 6

OpenMP: Hello World #include <stdio.h> #include <omp.h> int main() { // Do this part in parallel #pragma omp parallel { int ID = omp_get_thread_num(); OpenMP include file (Header file) Parallel region with default number of threads } } return 0; printf(" Hello(%d) ", ID); printf(" World(%d) \n", ID); Runtime library function to return a thread ID End of the Parallel region 7

OpenMP: Hello World #include <stdio.h> #include <omp.h> int main() { omp_set_num_threads(4); OpenMP include file (Header file) Sets the number of threads for the Parallel Region // Do this part in parallel #pragma omp parallel { int ID = omp_get_thread_num(); Starts the Parallel region with 4 threads End of the Parallel region } } return 0; printf(" Hello(%d) ", ID); printf(" World(%d) \n", ID); Runtime library function to return a thread ID 8

OpenMP: Hello World All OpenMP directives begin: #pragma #include <stdio.h> #include <omp.h> int main() { omp_set_num_threads(4); printf printf printf printf // Do this part in parallel #pragma omp parallel { printf( "Hello, World!\n" ); } return 0; } OpenMP is used for computationally intensive work on each node (core) 9

OpenMP Overview How do threads interact? OpenMP is a multi-threading, shared address model for parallelism. Threads communicate by sharing variables (Within the same process). Note: Any unintended sharing of data may cause race conditions: race condition occurs when the program s outcome changes as the threads are scheduled differently. To control race conditions: Use synchronization to protect data conflicts. Synchronization is expensive so: Change how data is accessed to minimize the need for synchronization. 10

Summary In addition to a parallel construct, You may need the following runtime library routines: void omp_set_num_threads(); int omp_get_num_threads(); int omp_get_thread_num(); 11