Skip to content
Snippets Groups Projects
Commit ce6f840f authored by Martin Schroschk's avatar Martin Schroschk
Browse files

Collect MPI usage issues on new page

* Resolves #291, #270, and #163
parent ca8766e1
No related branches found
No related tags found
2 merge requests!707Automated merge from preview to main,!630Mpi errors
# Known MPI-Usage Issues
This pages holds known issues observed with MPI and concrete MPI implementations.
## R Parallel Library on Multiple Nodes
Using the R parallel library on MPI clusters has shown problems when using more than a few compute
nodes. The error messages indicate that there are buggy interactions of R/Rmpi/OpenMPI and UCX.
Disabling UCX has solved these problems in our experiments.
We invoked the R script successfully with the following command:
```
console mpirun -mca btl_openib_allow_ib true --mca pml ^ucx --mca osc ^ucx -np 1 Rscript
--vanilla the-script.R
```
where the arguments `-mca btl_openib_allow_ib true --mca pml ^ucx --mca osc ^ucx` disable usage of
UCX.
## MPI Function `MPI_Win_allocate`
The function `MPI_Win_allocate` is a one-sided MPI call that allocates memory and returns a window
object for RDMA operations (ref. [man page](https://www.open-mpi.org/doc/v3.0/man3/MPI_Win_allocate.3.php)).
> Using MPI_Win_allocate rather than separate MPI_Alloc_mem + MPI_Win_create may allow the MPI implementation to optimize the memory allocation.
> (Using advanced MPI)
It was observed for at least for the `OpenMPI/4.0.5` module that using `MPI_Alloc_mem` in
conjunction with `MPI_Win_create` instead of `MPI_Win_Allocate` leads to segmentation faults in the
calling application. To be precise, the segfaults occurred at partition `romeo` when about 200 GB
per node where allocated. In contrast, the segmentation faults vanished when the implementation was
refactored to call the `MPI_Win_allocate` function.
......@@ -80,6 +80,7 @@ nav:
- Compilers and Flags: software/compilers.md
- GPU Programming: software/gpu_programming.md
- Mathematics Libraries: software/math_libraries.md
- MPI Usage Issues: software/mpi_issues.md
- Debugging: software/debuggers.md
- Software Engineering Tools:
- MPI Error Detection: software/mpi_usage_error_detection.md
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment