Lead Data Engineer

We are seeking an experienced Lead Data Engineer to spearhead our data engineering efforts for the Marketing Growth team. In this crucial role, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support our marketing analytics and data science initiatives.

Key Responsibilities:

• Lead the design and implementation of robust data pipelines that ingest, transform, and load data from various sources into our data lake and real-time systems.
• Collaborate with data scientists and business stakeholders to understand their data requirements and translate them into efficient data engineering solutions.
• Ensure data quality by implementing data validation, testing, and monitoring processes.
• Manage the integration of third-party data sources and APIs into our data ecosystem.
• Oversee the maintenance and optimization of existing data pipelines, addressing performance issues and implementing improvements as needed.
• Provide technical leadership and mentorship to junior data engineers, fostering a culture of continuous learning and knowledge sharing.
• Collaborate with platform teams to frame feature requirements and migrate between managed platform offerings and data products.

Requirements

The Must-Haves:

• At least Bachelor's in Computer Science, Information Technology, or a related field.
• Minimum of 5 years of experience in data engineering roles, with a strong background in designing and implementing scalable data pipelines and architectures.
• Proven leadership experience, with the ability to mentor and guide junior team members.
• Strong communication and collaboration skills, with the ability to effectively work with cross-functional teams, including data scientists, analysts, and business stakeholders.
• Proficient at least one of the programming languages such as Python, Scala, or Java, and familiarity with big data technologies like Apache Spark, Apache Kafka, and Apache Airflow.
• Strong understanding of data modeling, ETL/ELT processes, and data warehousing concepts.
• Experience with real-time data processing and streaming data architectures.
• Excellent problem-solving and analytical skills, with the ability to break down complex problems into manageable tasks.

The Nice-to-Haves:

• Candidates with expertise in streaming data using frameworks like Apache Flink will have an added advantage.
• Extensive experience with cloud-based data platforms and services such as AWS is highly valued.