About: Desynchronization and Wave Pattern Formation in MPI-Parallel and Hybrid Memory-Bound Programs

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Desynchronization and Wave Pattern Formation in MPI-Parallel and Hybrid Memory-Bound Programs Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : covidontheweb.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
title	Desynchronization and Wave Pattern Formation in MPI-Parallel and Hybrid Memory-Bound Programs
Creator	Afzal, Ayesha Hager, Georg Wellein, Gerhard
source	PMC
abstract	Analytic, first-principles performance modeling of distributed-memory parallel codes is notoriously imprecise. Even for applications with extremely regular and homogeneous compute-communicate phases, simply adding communication time to computation time does often not yield a satisfactory prediction of parallel runtime due to deviations from the expected simple lockstep pattern caused by system noise, variations in communication time, and inherent load imbalance. In this paper, we highlight the specific cases of provoked and spontaneous desynchronization of memory-bound, bulk-synchronous pure MPI and hybrid MPI+OpenMP programs. Using simple microbenchmarks we observe that although desynchronization can introduce increased waiting time per process, it does not necessarily cause lower resource utilization but can lead to an increase in available bandwidth per core. In case of significant communication overhead, even natural noise can shove the system into a state of automatic overlap of communication and computation, improving the overall time to solution. The saturation point, i.e., the number of processes per memory domain required to achieve full memory bandwidth, is pivotal in the dynamics of this process and the emerging stable wave pattern. We also demonstrate how hybrid MPI-OpenMP programming can prevent desirable desynchronization by eliminating the bandwidth bottleneck among processes. A Chebyshev filter diagonalization application is used to demonstrate some of the observed effects in a realistic setting.
has issue date	2020-05-22(xsd:dateTime)
bibo:doi	10.1007/978-3-030-50743-5_20
has license	cc-by
sha1sum (hex)	b2e4110e86f39e94ad7e772ae2a253ff2a114d62
schema:url	https://doi.org/10.1007/978-3-030-50743-5_20
resource representing a document's title	Desynchronization and Wave Pattern Formation in MPI-Parallel and Hybrid Memory-Bound Programs
has PubMed Central identifier	PMC7295361
schema:publication	High Performance Computing
resource representing a document's body	covid:b2e4110e86f39e94ad7e772ae2a253ff2a114d62#body_text
is schema:about of	named entity 'bottleneck' named entity 'noise' named entity 'diagonalization' named entity 'Chebyshev filter' named entity 'process' named entity 'deviations' named entity 'prediction' named entity 'Programs' named entity 'phases' named entity 'prevent' named entity 'OpenMP' named entity 'wave' named entity 'solution' named entity 'expected' named entity 'communication' named entity 'system' named entity 'imbalance' named entity 'caused' named entity 'pattern' named entity 'Wave' named entity 'bandwidth' named entity 'Chebyshev filter' named entity 'memory-bound' named entity 'Pattern Formation' named entity 'Memory-Bound' named entity 'communication' named entity 'applications' named entity 'CPUs' named entity 'bandwidth' named entity 'dual-socket' named entity 'MPI' named entity 'optimization' named entity 'Chebyshev polynomials' named entity 'execution time' named entity 'periodic boundary conditions' named entity 'MPI' named entity 'MPI' named entity 'vector operations' named entity 'quantum physics' named entity 'inter-process communication' named entity 'MPI' named entity 'memory bound' named entity 'nonlinear' named entity 'single-threaded' named entity 'bandwidth' named entity 'message size' named entity 'Omni-Path' named entity 'MPI' named entity 'memory-bound' named entity 'wavefront' named entity 'MPI' named entity 'MPI' named entity 'MPI' named entity 'MPI' named entity 'OpenMP' named entity 'pattern formation' named entity 'bandwidth' named entity 'MPI' named entity 'memory-bound' named entity 'trailing edge' named entity 'MPI' named entity 'Hyper-Threading' named entity 'MPI' named entity 'memory bandwidth' named entity 'MPI' named entity 'kernel' named entity 'constant time' named entity 'boundary conditions' named entity 'topological materials' named entity '1.2' named entity 'Emmy' named entity 'MPI' named entity 'latency' named entity 'lock-step' named entity 'wall-clock time' named entity 'MPI' named entity 'Intel Xeon' named entity 'electronic structure' named entity 'deterministic' named entity 'MPI' named entity 'bandwidth'

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2025 OpenLink Software