Baseten’s Inference Stack team builds the distributed runtime that powers large-scale LLM inference across our platform. We operate at the intersection of distributed systems, model performance, infrastructure, and developer experience. We enable customers to deploy and operate cutting-edge LLM models with industry-leading performance, scalability, reliability, and ease of use
As a Software Engineer on the Inference Stack team, you’ll work across the stack - from the developer experience customers use to deploy models, the libraries used for features like tool calling and reasoning, all the way down to the systems we use to orchestrate deployments in Kubernetes and route traffic efficiently
This is an ideal role for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users
EXAMPLE INITIATIVES
Blog Posts
Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference
Work across the stack, from customer-facing features to low-level infrastructure components
Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management
Improve the reliability, scalability, and usability of our inference stack
Collaborate closely with Model Performance engineers to make new inference optimizations broadly available to customers and easy to configure
Help define best practices around testing, release automation, benchmarking, and operational excellence
Debug complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads
Make thoughtful engineering tradeoffs balancing performance, reliability, operational simplicity, and developer experience
Own projects end-to-end: from architecture and implementation through deployment, monitoring, and iteration based on customer feedback