New models support. Support transformer-based retrieval, text-generation, and multimodal models in our inference infrastructure, from weight loading, request scheduling and KV-cache management to support in API Gateway
GPU kernels migration to CuTe DSL. Port our in-house CUDA kernels to NVIDIA's CuTe DSL so they run on GB200 today and are portable to Vera Rubin racks tomorrow
Rust-native serving runtime. Develop our internal Rust-based inference server to solve all Python pains and keep up with rapidly growing traffic
Performance optimisation. Profile and fix bottlenecks from network ingress through continuous batching and GPU kernel interleaving
Reliability and observability. Build dashboards, alerts, and automated remediation so we catch regressions before users do. Respond to and learn from production incidents
Deep experience with GPU programming and performance work (CUDA, Triton, CUTLASS, or similar). Any other deep systems programming experience is a plus
You understand modern LLM architectures and are able to bring them up reliably in a production environment
You've built and operated production distributed systems under real load - ideally performance-critical ones
Comfortable working across languages and layers: Rust for the serving runtime, Python for model code, CUDA/CuteDSL for kernels
You own problems end-to-end. You can read a research paper on Monday, write a kernel on Wednesday, and debug a production incident on Friday
Self-directed. You do well in fast-moving environments where the path forward isn't laid out for you
ML compilers and framework internals: PyTorch internals, torch.compile, custom operators
Distributed GPU communication: NCCL, NVLink, InfiniBand, RDMA libraries, model/tensor parallelism
Low-precision inference: INT8/FP8/FP4 quantization, mixed-precision serving
Profiling and debugging tools: Nsight Compute/Systems, CUDA-GDB, PTX/SASS analysis
Container orchestration: Kubernetes, GPU scheduling, autoscaling inference workloads