Data Engineer

About your tasks:

• Design, build, and maintain data pipelines using Microsoft Azure technologies, ensuring data is collected, stored, and processed efficiently.
• Implement observability practices to monitor data workflows and system health, ensuring data quality and system reliability.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize and scale our data infrastructure to handle growing data volumes and new data sources.
• Document data flows and system architecture, ensuring transparency and ease of maintenance.

Requirements

About you:

• Expertise in Azure Databricks, Azure Data Factory, Azure SQL Database, Event-Hubs and other Azure-based data services.
• Proven experience in building, managing and monitor ETL/ELT pipelines for Batch and Realtime purposes, with a focus on performance and scalability. Experience in Apache Airflow and dbt is a plus.
• Proficiency in SQL, Spark, Python, or other programming languages commonly used for data engineering tasks and API integration.
• Familiarity with tools like Terraform, Azure DevOps Deployment Pipelines, Azure Monito...