Lead Data Engineer ID71008

Agileengine

≈ 174,4 тыс.–196,4 тыс. ₽ · 2,2 тыс.–2,5 тыс. $ · 1,9 тыс.–2,1 тыс. €

7 000 000 – 7 885 000 COP

🌍 Удалённо

Senior

Полная занятость

🌐 Глобал

Продуктовая

Tech

Описание вакансии

Обязанности

•Create Airflow DAG batch workflows orchestration and dependencies

•Define partitioning file formats schema and data types

•Design ETL pipelines for data extraction transformation and validation

•Implement near real time processing for OLAP workloads

•Lead developers and align engineering practices

•Operate AWS data lake architecture with S3 Athena and EKS Kubernetes

•Partner with DevOps team on functional and non functional requirements

•Review pull requests and enforce code quality standards

Условия

•Annual learning budget

•Flexible hours

•Internal technical talks

•Mentorship

•People focused support

•Remote work

•Well-being programs

Технологии: Amazon EKS, Amazon S3, Apache Airflow, Apache Parquet, Athena, BigQuery, ClickHouse, Data Lake, Data Partitioning, Data Validation, Docker, ETL, FastAPI, Flask, GraphQL, Kubernetes, OLAP, ORC, Object storage, PostgreSQL, Presto, Python, REST, Schema Design, Snowflake, Spark SQL, Trino