MIMD (Multiple Instructions, Multiple Data)
Multithreading (MIMD parallelism) refers to the capability of a single processor to concurrently execute multiple independent instruction streams of a single executable called threads using a method called time sharing. Threads are only bounded by data memory (DMEM), and each thread has a separate data-path. Furthermore, there exist two types of threads:
- Software threads: managed by the operating system or a runtime environment within an application.
- Hardware threads: a physical feature of certain CPU architectures that allow multiple threads to be executed simultaneously within a single physical core.
One model for multithreading is the fork-join model, where the program starts serial, with a master thread “forking” the work across all threads during parallel segments, to then synchronize (join) and terminate all threads except itself after the parallel segment is finished.
Note! load balancing is crucial, and load should be distributed evenly so that all threads finish almost at the same time. The parallel segment only terminates once the last thread finishes its work. If two or more threads depend on the same critical resource at the same time, they might be unable to proceed with their execution resulting in a deadlock, where the whole program will crash even if only a single thread crashes.
In the fork-join model,
Examples
Multithreaded Supercomputer
Multithreading in C
The go-to library for multithreading in C, C++ or Fortran is OpenMP. We use it below to