Parallel Computing Final Exam Review
Need OS and compiler support for...
OpenMP
Need OS support for...
Pthreads
What is the smallest type?
a byte
Define grid.
a collection of blocks organized in a 1d, 2d, or 3d Cartesian geometry
Define block.
a collection of threads organized in a 1d, 2d, or 3d Cartesian geometry
Define kernel.
a function declared with a specifier __global__
Besides a stack, what does a thread have?
a program counter
Define cluster.
a set of computers connected by a network to form a single computing system
What is a void pointer?
a super type of all pointer types that can point to any type
What is a complete type?
a type that has a size
Define unsigned type.
a type that includes only nonnegative integers
Define signed type.
a type that includes positive and negative integers
What does MPI_Reduce do?
adds all values given by each process
What does malloc do?
allocates space on the heap for some object
Define process.
an independently running instance of a program
What does reduction do?
applies some operation across all iterations for some variables
Describe block distribution in MPI.
array is divided up into section for each process
What is mutual exclusion?
at most, one thread is in the critical section at any time
What does #pragma omp barrier do?
blocks all threads in a team until every thread in the team has reached the barrier
What does pthread_join do?
blocks until the specified thread terminates, then deallocates the thread resources
Does a process have security information or state information?
both
Define supercomputer.
clusters of multi-core nodes
Define semaphore.
concurrency primitive with a state which is a single nonnegative integer that is the semaphore's value
In what memory space does __constant__ reside in?
constant memory
What does MPI_Send do? (may do 1 of 2 things)
copies message into a system buffer or forces the sender to wait until a receive operation is issued
What does #pragma omp parallel do?
creates a parallel region in which a team of threads is created
What does free do?
deallocates an object previously place don the heap by malloc
What does ordered do?
declares that an ordered construct may occur in loop body
What does pthread_cond_destroy do?
destroys the previously initialized condition variable
Are new threads created statically or dynamically?
dynamically
Describe the message-passing programming model.
each node runs its own process with independent variables and processes communicate through function
How do mutexes work?
each thread obtains the lock before entering the critical section and releases the lock when it is done
What does MPI_Alltoall do?
every process sends distinct buffers to all others
Define statically typed.
every variable and expression has a type that is known at compile time
True or false: A process can only run one thread at any time.
false
True or false: A process has public address space.
false
True or false: Blocks must execute in a certain order.
false
When alone, in what memory space does __device__ reside?
global memory
Does a process have stack or heap memory?
heap memory
Describe fairness in a critical section.
if a thread is trying to enter the critical section then eventually it will succeed
What is MPI_ANY_SOURCE?
ignores the source of a receive
What is MPI_STATUS_IGNORE?
ignores the status of a receive and can be used as the last argument of MPI_Recv
What is MPI_ANY_TAG?
ignores the tag of a receive and can be used as the last argument of MPI_Recv
What does ptherad_cond_init do?
initializes a condition variable
What does invoking a kernel function do?
instantiates a grid executing on a device
What does MPI_Gather do?
inverse of MPI_Scatter, one process received from all others
What does MPI_Scatter do?
like MPI_Bcast but every receiver gets a different block of elements
What does private do?
makes a shared variable private for the duration of the loop
What does lastprivate do?
makes a variable private and copies the final value of the variable in the last iteration back to the shared variable at the end
What does firstprivate do?
makes a variable private and initializes it every thread
What does num_thread(expr) do?
makes the team have a specified number of threads where expr is the number of threads
Define multi-core.
multiple processors on the same chip that share resources
Define mutex.
mutual exclusion lock
Can a void pointer be dereferenced?
no
What should a critical section of code have? (4 things)
no deadlocks, no unnecessary delay, mutual exclusion, and fair
What does MPI_Barrier do?
no process can leave until all have entered
What does MPI_ANY_SOURCE introduce into a program?
nondeterminism
Describe cyclic distribution in MPI.
often done with for loop incrementing by nprocs
How many threads are there when a program starts?
one
Describe the shared variable programming model.
one program spawns multiple threads and the threads communicate by writing to and reading from shared variables
What does schedule do?
options to control how iterations are distributed to threads; can be static, dynamic, or guided
Are local variables shared or private?
private
What does pthread_cond_wait do/
releases the lock on the mutex, goes to sleep, and tries to regain the lock on the mutex when it wakes up
What does nowait do?
removes the barrier at the end of the loop
What does MPI_Allreduce do?
same as MPI_Reduce but no root
What does MPI_Bcast do?
sends a message from one process to all others
Are global variables shared or private?
shared
How do we determine efficiency?
speedup / number of processes
Does a thread have stack or heap memory?
stack
Is C statically or dynamically typed?
statically
Does a process need resources from the OS or from the compiler?
the OS
Define pointer.
the address of a memory location
What does an ampersand represent (&x)?
the address of a variable
Is __device__ callable from the device or the host?
the device
Is __device__ executed on the device or the host?
the device
Is __global__ executed on the device or the host?
the device
Is __global__ callable from the device or the host?
the host
Is __host__ callable on the device or the host?
the host
Is __host__ executed on the device or the host?
the host
What happens if a thread tries to obtain the mutex lock while another thread already owns it?
the thread trying to obtain the lock will block until it becomes available
What does an asterisk represent (*x)?
the value stored in a variable
How are __device__ and __constant__ accessible from the host?
through library functions
How do we determine speedup?
time taken serial / time taken parallel
When is a void pointer necessary?
to design generic functions
True or false: __device__ is accessible from all threads within the grid.
true
If unspecified, a type is...
unsigned
When is MPI_ANY_SOURCE used?
used when the receive does not know what send function it will receive from
How are new threads created?
using pthread_create
What does __global__ return?
void
What are three synchronization mechanisms?
wait until another thread terminates, block until a lock becomes available then release the lock, wait until some condition becomes true
What does pthread_cond_broadcast do?
wakes up all threads waiting on cond
What does pthread_cond_signal do?
wakes up one or more threads waiting on a condition
When is MPI block distribution used?
when a program needs to access neighbors in array
What is unnecessary delay?
when a thread that is trying to enter the critical section when no one else is in the critical section encounters a delay
When is MPI cyclic distribution used?
when chunks of data which take a long time to run are next to each other
When does MPI_Recv unblock?
when the receive buffer is full
Define data race.
when two or more threads can access the same memory concurrently and at least one of the accesses is a write
Can multiple threads exist within a single process?
yes
Does a process need to have environment variables?
yes
Is __constant__ accessible from all threads in the grid?
yes
