Staff Software Engineer, Model Infrastructure

Harvey·San Francisco·22 июля

Staff Software Engineer, Model Infrastructure

🏢 ОфисSeniorПолная занятость

Зарплата не указана

Вилки нет, про деньги придётся договариваться с нуля.

Нажмите на сигнал, чтобы увидеть, на чём он основан

Наша компания

WHY HARVEY At Harvey, we’re transforming how legal and professional services operate. By combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we’re reshaping how critical knowledge work gets done for decades to come.

О роли

This is a rare chance to help build a generational company at a true inflection point. With 2400+ customers in 70+ countries, strong product-market fit, and world-class investor support, we’re scaling fast and defining a new category in real time. The work is ambitious, the bar is high, and the opportunity for growth — personal, professional, and financial — is unmatched. Our team moves fast, take

Чем предстоит заниматься

Lead the design and implementation of Harvey's Model Infrastructure platform

Build systems to ensure high availability, low latency, and operational excellence for AI inference

Design and improve Harvey's Unified Model Controller (UMC) and Model Selector platform to automatically detect model degradations and intelligently route traffic based on reliability, latency, quality, compliance, and cost

Develop systems for model provisioning, capacity management, failover, and traffic engineering across multiple AI providers

Integrate new model providers and maintain provider APIs and SDKs, enabling Harvey to rapidly adopt emerging frontier models

Improve observability through health dashboards, alerting, token usage analytics, cost reporting, and end-to-end telemetry

Partner with Product Engineering to support model launches, experimentation, and proactive monitoring of production AI workloads

Drive infrastructure efficiency through capacity planning, utilization optimization, and cost visibility

Collaborate with AI Research to build the infrastructure foundation for future model evaluation, training, and deployment

Lead cross-functional technical initiatives and mentor engineers across the organization

WHAT YOU'LL BUILD

You'll help build the core platform behind Harvey's AI capabilities, including

Наши требования

Experience with AI infrastructure, LLM serving, or machine learning platforms

Experience with model routing, inference gateways, or policy-based serving systems

Experience working with OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, or open-source LLMs

Experience with Kubernetes, cloud infrastructure, and service mesh technologies

Experience with large-scale observability and SRE best practices

Experience with data infrastructure technologies such as Kafka, Spark, Flink, Airflow, or Iceberg

Familiarity with GPU infrastructure or model training platforms

7+ years of software engineering experience building large-scale distributed systems

Experience designing and operating highly available production services

Strong programming skills in Go, Java, Python, Rust, or C++

Deep understanding of distributed systems, cloud infrastructure, networking, and observability

Experience leading technical projects across multiple engineering teams

Ability to balance long-term architecture with pragmatic execution

Strong communication and collaboration skills

Passion for building foundational platforms that enable other engineering teams

Дополнительно

Model health monitoring

Automated failover and recovery

Capacity provisioning

Operational tooling and incident automation

Policy-based model routing

Intelligent Model Selector

Traffic management

Reliability and latency optimization

Multi-provider architecture

API and SDK integrations

OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, and future providers

Rapid adoption of new frontier models

Token usage analytics

Cost attribution

Latency and reliability dashboards

Capacity forecasting

Utilization optimization

Infrastructure supporting model evaluation

Model deployment and operations

Future model training platform

Agent infrastructure and CcaaS

$236,000 - $290,000 USD

Harvey

San Francisco

ГрейдSenior

ЗанятостьПолная занятость

РегионСША

ФорматОфис

ИсточникСкрыто

Опубликовано22 июля

Все вакансии компании

AI-помощник

под эту вакансию

Войди, чтобы AI оценил твоё соответствие вакансии и написал сопроводительное письмо

Мы против мошенников на площадке: если тебя просят заплатить, продиктовать код или установить непонятное приложение, прекращай общение и сразу пиши нам (чат с основателем или форма обратной связи).