Data Engineer

Sabenza It

Johannesburg, South Africa

≈ 182,3 тыс.–188,6 тыс. ₽ · 2,3 тыс.–2,4 тыс. $ · 2 тыс.–2,1 тыс. €

38 667 – 40 000 ZAR

🏢 Офис

Без опыта

Контракт

Продуктовая

Tech

Описание вакансии

Обязанности

•Architect and develop batch and real time data pipelines

•Build data quality validation reconciliation and auditing frameworks

•Collaborate with data scientists architects analysts and software engineers

•Design and optimize Azure based data platforms

•Develop APIs and services for secure real time data access

•Enforce governance and security best practices

•Implement DataOps automation orchestration testing and CI CD

•Implement infrastructure-as-code

•Improve platform performance scalability reliability and observability

•Ingest transform and deliver structured semi structured and unstructured data

•Manage data lineage and metadata

•Own end-to-end data lifecycle

•Participate in agile ceremonies

•Profile data and perform root cause analysis

•Support complex production issues

Технологии: Ab Initio, Apache Spark, Azure Data, Azure Data Factory, Azure Event, Azure Event Hub, Azure Synapse, CI/CD, Data Factory, Data Lakes, Data Lineage, Data Modelling, Data Warehousing, DataOps, Databricks, ELT, ETL, Event Hub, Infrastructure as Code, Kafka, Microsoft Azure, Microsoft Fabric, PySpark, Python, SQL, “as-code”