Proficiency in Python, TypeScript, or other programming languages used in AI and test automation
Proven skill in designing or applying rubric-based evaluation, grading against set criteria, or building structured scoring frameworks
Direct experience with LLM evaluation frameworks and benchmarking tools such as LangSmith, Confident AI, etc
Knowledge of the GenAI stack and solutions including Retrieval Augmented Generation (RAG)
LLMs: Azure OpenAI, Vertex AI, ChatGPT Enterprise or similar
High attention to detail and ability to notice subtle patterns or inconsistencies (such as data hallucinations or logic errors) that others might miss
Advanced written communication skills, especially for documenting nuanced observations and feedback
Experience with Cloud platforms (Azure, GCP, AWS)
Thorough understanding of DevOps/automation/CI/CD tools: GitHub, Terraform
Comprehension with AI ethics, risk management, and data governance
Additional Information - We Take Care of Our People
As a distributed company, diversity drives our identity. Whether you’re looking to launch a new career or grow an existing one, Elastic is the type of company where you can balance great work with great life. Your age is only a number. It doesn’t matter if you’re just out of college or your children are; we need you for what you can do
We strive to have parity of benefits across regions and while regulations differ from place to place, we believe taking care of our people is the right thing to do
Competitive pay based on the work you do here and not your previous salary
Health coverage for you and your family in many locations
Ability to craft your calendar with flexible locations and schedules for many roles
Generous number of vacation days each year
Increase your impact - We match up to $2000 (or local currency equivalent) for financial donations and service
Up to 40 hours each year to use toward volunteer projects you love
Embracing parenthood with minimum of 16 weeks of parental leave
Please see here for our Privacy Statement