Lead Data Engineer ID71008

Agileengine

≈ 418,4 тыс.–519,4 тыс. ₽ · 5,2 тыс.–6,5 тыс. $ · 4,6 тыс.–5,7 тыс. €

26 583 – 33 000 BRL

🌍 Удалённо

Senior

Полная занятость

🌐 Глобал

Продуктовая

Tech

Описание вакансии

Обязанности

•Build Airflow DAG workflows

•Collaborate with DevOps team

•Define partitioning and file format strategy

•Deploy and manage Kubernetes workloads

•Design ETL pipelines

•Design data lake architecture

•Implement near real time processing

•Lead developers and align engineering practices

•Manage S3 based data lake

•Orchestrate scheduled batch workflows

•Review pull requests and enforce code quality

•Use Athena for querying

Условия

•Flexible hours

•Internal technical talks

•Learning budget

•Mentorship

•Performance and compensation reviews

•Remote work

•Well-being programs

Технологии: AWS, Amazon Athena, Amazon S3, Apache Airflow, Apache Spark, BigQuery, ClickHouse, Data Engineering, Docker, EKS, ETL, FastAPI, Flask, GraphQL, Kubernetes, Near-real-time, Near-real-time Processing, OLAP, ORC, Parquet, Partitioning, PostgreSQL, Presto, Python, REST, Real-time Processing, Schema Design, Snowflake, Spark SQL, Time processing, Trino