Технологии: C++, CPU Optimization, CUDA, CUDA graph, Communication overlap, Data parallelism, DeepSpeed, IO optimization, Imitation Learning, Long Context, Long Context Training, Megatron, Mixed Precision, Mixed-precision training, Mixture of Experts, NVIDIA Nsight, Nsight Systems, Nvidia NSight Systems, Offline Reinforcement Learning, Online Reinforcement Learning, Performance Profiling, Pipeline parallelism, PyTorch, PyTorch Profiler, Python, RDMA, Reinforcement Learning, Rust, Tensor Parallelism, Triton