Harvey·San Francisco·5 дн назад
Senior Software Engineer, Model Infrastructure
193 400 – 290 000 $ в год
на 17% выше медианы рынка
≈ 1,4 млн–2 млн ₽
🏢 ОфисSeniorПолная занятость
Вилки нет, про деньги придётся договариваться с нуля.
Наша компания
WHY HARVEY At Harvey, we’re transforming how legal and professional services operate. By combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we’re reshaping how critical knowledge work gets done for decades to come.
О роли
This is a rare chance to help build a generational company at a true inflection point. We have strong product-market fit and world-class investor support. We’re scaling fast and defining a new category in real time. The work is ambitious, the bar is high, and the opportunity for growth — personal, professional, and financial — is unmatched. Our team moves fast, takes ownership, and is deeply commi
Чем предстоит заниматься
Lead the design and implementation of Harvey's Model Infrastructure platform
Build systems to ensure high availability, low latency, and operational excellence for AI inference
Design and improve Harvey's Unified Model Controller (UMC) and Model Selector platform to automatically detect model degradations and intelligently route traffic based on reliability, latency, quality, compliance, and cost
Develop systems for model provisioning, capacity management, failover, and traffic engineering across multiple AI providers
Integrate new model providers and maintain provider APIs and SDKs, enabling Harvey to rapidly adopt emerging frontier models
Improve observability through health dashboards, alerting, token usage analytics, cost reporting, and end-to-end telemetry
Partner with Product Engineering to support model launches, experimentation, and proactive monitoring of production AI workloads
Drive infrastructure efficiency through capacity planning, utilization optimization, and cost visibility
Collaborate with AI Research to build the infrastructure foundation for future model evaluation, training, and deployment
Lead cross-functional technical initiatives and mentor engineers across the organization
Наши требования
Experience with AI infrastructure, LLM serving, or machine learning platforms
Experience with model routing, inference gateways, or policy-based serving systems
Experience working with OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, or open-source LLMs
Experience with Kubernetes, cloud infrastructure, and service mesh technologies
Experience with large-scale observability and SRE best practices
Experience with data infrastructure technologies such as Kafka, Spark, Flink, Airflow, or Iceberg
Familiarity with GPU infrastructure or model training platforms
4+ years of software engineering experience building large-scale distributed systems
Experience designing and operating highly available production services
Strong programming skills in Go, Java, Python, Rust, or C++
Deep understanding of distributed systems, cloud infrastructure, networking, and observability
Experience leading technical projects across multiple engineering teams
Ability to balance long-term architecture with pragmatic execution
Strong communication and collaboration skills
Passion for building foundational platforms that enable other engineering teams
Дополнительно
Model Reliability & Operations
Model health monitoring
Automated failover and recovery
Capacity provisioning
Operational tooling and incident automation
Unified Model Controller (UMC)
Policy-based model routing
Intelligent Model Selector
Traffic management
Reliability and latency optimization
Provider Platform
Multi-provider architecture
API and SDK integrations
OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, and future providers
Rapid adoption of new frontier models
Observability & Cost Platform
Token usage analytics
Cost attribution
Latency and reliability dashboards
Capacity forecasting
Utilization optimization
AI Platform Foundation
Infrastructure supporting model evaluation
Model deployment and operations
Future model training platform
Agent infrastructure and CcaaS
$193,400 - 290,000 USD