Описание вакансии
Обязанности
•Architect and govern DAG based orchestration
•Build and manage S3 backed data lake pipelines
•Define partitioning and file format strategy
•Design and own ETL pipelines
•Design near real time processing for OLAP workloads
•Drive AWS data stack including Athena and EKS
•Ensure maintainability with AI assisted development tools
•Lead engineering team and set technical standards
•Partner with DevOps to clarify requirements
•Review pull requests and enforce code quality
Условия
•Annual learning budget
•Flexible hours
•Mentorship
•People focused support
•Remote work
•Techtalks
•Well-being programs
Технологии: Airflow, Amazon S3, Apache Spark, Apache Trino, Athena, BigQuery, ClickHouse, DAG Workflow, Data Lake, Data Validation, Docker, EKS, ETL, FastAPI, File format, File format strategy, Flask, GraphQL, Kubernetes, Near-real-time, Near-real-time Processing, ORC, Object storage, Parquet, Partitioning Strategy, PostgreSQL, Presto, Python, REST, Real-time Processing, Schema Design, Snowflake, Spark SQL, Time processing, Trino