Skip to content
Snippets Groups Projects
Commit 951ed682 authored by Jan Frenzel's avatar Jan Frenzel
Browse files

Apply 1 suggestion(s) to 1 file(s)

parent c9100fa4
No related branches found
No related tags found
3 merge requests!392Merge preview into contrib guide for browser users,!378Merge p in m,!367Update the distributed_training.md Pytorch section
......@@ -158,7 +158,7 @@ achieve true parallelism due to the well known issue of Global Interpreter Lock
Python. To work around this issue and gain performance benefits of parallelism, the use of
`torch.nn.DistributedDataParallel` is recommended. This involves little more code changes to set up,
but further increases the performance of model training. The starting step is to initialize the
process group by calling the `torch.distributed.init_process_group()` using the appropriate backend
process group by calling the `torch.distributed.init_process_group()` using the appropriate back end
such as NCCL, MPI or Gloo. The use of NCCL as back end is recommended as it is currently the fastest
back end when using GPUs.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment