CS 33361 Chapter 9b Review

What languages do and do not support monitors?

Do support monitors: Ada, Java Visual Basic, C# Do not support monitors: Python, C, C++

Communication Patterns - Broadcast

During a broadcast, one process sends the same data to all processes.

Communication Patterns - Multicast

During a multicast: • One process sends the same data to many processes • Many processes send the same data to many processes • One-to-many vs many-to-many

Granularity

Property of locks and measured as the amount of data the lock is protecting.

what can semaphores be used for?

Semaphores can be used to implement guards on the code that access shared data structures.

What are the only two operations that semaphores can have?

Semaphores have only two operations, wait and release (originally called P and V by Dijkstra)

What is the Global Interpreter Lock (GIL)?

The Python GIL is a mutex lock. • The Global Interpreter Lock is used to ensure the entire interpreter is locked to only one thread at a time. • The Global Interpreter Lock only gets invoked during the interpretation of a command for execution. • As such, it disproportionally affects CPU bound instructions.

Deadlock

The situation when each of at least two tasks is waiting for a lock the other task holds.

mutex locks are semaphores with a maximum thread count of 1. True or false?

True

• Monitors were born out of the shift to abstract data typing. True or false? --- Monitors are abstract data types for shared data. --- Monitors encapsulate the shared data and its operation to restrict access.

True

What is a rendezvous?

When a sender task's message is accepted by a receiver task, the actual message transmission is called a rendezvous.

•The more locks a program uses, the _____ the overhead associated with the usage.

higher

Message Passing is for communication among _______________.

processes (not threads)

Amdahl's Law simply states that:

the overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used. As such, if: • T = Total time of serial execution • B = Total time of non-parallelizable part • T - B = Total time of parallelizable part when executed serially. ----------------- Then: T = B + (T - B)

•To support concurrent tasks with message passing, a language needs:

• A mechanism to allow a task to indicate when it is willing to accept messages • • A way to remember who is waiting to have its message accepted and some "fair" way of choosing the next message • • When a sender task's message is accepted by a receiver task, the actual message transmission is called a rendezvous

Amdahl's Law

• Amdahl's Law can be used to calculate how much a computation can be sped up by running it in parallel. - A program which can be parallelized can be split up into two distinct parts: • The part which can be parallelized • The part which cannot be parallelized

The difference between semaphores and mutexes?

• An object's mutex can only be acquired by one thread at a time. • An object's semaphore contains a thread counter, so it can be acquired by multiple threads simultaneously.

Communication Patterns - Scatter/Gather

• During a scatter, one process sends different data to all processes. • • During a gather, one process receives results from all processes

Amdahl's Law Formula

• For serial executionL: T = B + (T - B) ---------- •What about for parallel execution? • If, N is the parallelization factor (which for us will often be the number of threads). Then: T(N) = B + (T - B) / N

•Fine Granularity:

• Larger number of locks, each protecting a fairly small amount of data. • Overhead: Results in increased overhead when a single process is accessing the data. • Contention: Results in better performance when multiple processes are running concurrently.

When does lock contention occur?

• Occurs when a process/thread attempts to acquire a lock held by another process/thread. • • The more "fine-grained" the available locks, the less likely this will occur.

•Course Granularity:

• Small number of locks, each protecting a large segment of data • Overhead: Results in less lock overhead when a single process is accessing the data. • Contention: Results in worse performance when multiple processes are running concurrently.

Memory bound

• The rate of execution is limited by the amount of memory available and the speed of memory access. • A process is considered memory bound if it processes large amounts of memory or spends much of its time waiting on the memory subsystems.

Cache bound

• The rate of execution is limited by the speed and size of the onboard cache. • A process if considered cache bound if it processes more data than fits in the cache.

CPU Bound

• The rate of execution is limited by the speed of the CPU. • A process is considered CPU bound if it spends most of its time simply using the CPU.

I/O Bound

• The rate of execution is limited by the speed of the I/O subsystem or time spent waiting for input. • A process if considered I/O bound if it spends most of its time handling I/O.

What is a mutex?

• mutex <- Mutual Exclusion --- A special mechanism for synchronizing thread access --- Attached to an object in a language --- Often referred to as a mutex lock •A mutex lock makes it possible to ensure that only one thread at a time has access to an object.

Semaphore

•A semaphore is a data structure consisting of a counter and a queue for storing task descriptors ----- A task descriptor is a data structure that stores all of the relevant information about the execution state of the task

Collective Communication

•Collective communication between multiple processes •Process group (collective) •Several processes logically grouped together • •Collective operations •Communication patterns broadcast, multicast, scatter/gather, ...

Pairwise Communication

•Communication between process pairs •Send/Receive or Put/Get •Synchronous or asynchronous • •Synchronous Communication •Multiple parties continually listen for and act upon replies from each other. •Real world example: Phone Conversation • •Asynchronous Communication •Parties do not actively listen for messages. •Real world example: Text Messaging

What is the dining philosophers problem?

•States that: •There are N philosophers are seated around a circular table •There is a single chopstick to their left •There is a single chopstick to their right •There exists a bowl of noodles in the center. • •The problem: •Philosopher's will spend some amount of time thinking then decide to eat. •They must possess the chopstick to their left and right in order to eat.

•So a mutex ensures only one thread at a time can work with a shared resource. •Attempts by other threads to gain access to the occupied resource will fail.

•This exposes two key features of a mutex / mutex lock: 1.) Like a binary value, a mutex can only exist in two possible states. --- Unlocked --- Locked 2.) The state cannot be controlled directly. • Programmers must work with mutex locks using tools within the language.

Summary of the powerpoint slides

Four synchronization methods •Mutex Locks •Semaphores •Monitors •Message Passing ----------- • Amdahl's Law governs how much speed up can be achieved through parallelization of code. ------------ • Python's Global Interpreter Lock makes multithreading with Python beneficial in only specific cases. --------- • Python's multiprocessing library makes concurrent Python code more beneficial but is more cumbersome when getting return values.