There is - https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/n... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

option on Feb 11, 2023 | parent | context | favorite | on: How to train large models on many GPUs? (2021)

There is - https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en...

Supports data, tensor, pipeline, sequence parallelisms, activation checkpointing, distributed optimizers, fused kernels and more.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact