Data Engineer - Fulfillment

The Data Engineering department at Tesco Technology is at the forefront of data processing within the retail and technology industry. This vital department handles a range of responsibilities, including:

• Analyzing order and delivery data to optimize logistics processes and enhance delivery efficiency.
• Managing critical data related to customer orders, suppliers, and products to ensure the seamless flow of our fulfillment operations.
• Upholding data integrity and security during the processing of order and delivery-related information.

As we continue to expand, we are actively seeking a skilled Data Engineer to join our team of analytics experts. In this role, you will take charge of expanding and refining our data and data pipeline architecture. Additionally, you will be instrumental in optimizing data flow and collection to cater to the needs of cross-functional teams.

Our ideal candidate is an experienced data pipeline builder and data enthusiast who relishes the opportunity to optimize data systems and construct them from the ground up. As a Data Engineer, you will collaborate closely with software developers, database architects, data analysts, and data scientists on various data-driven initiatives. You will play a crucial role in ensuring that the optimal data delivery architecture remains consistent across all ongoing projects.

This role calls for a high level of self-direction and the ability to effectively support the data requirements of multiple teams, systems, and products. If you are enthusiastic about the prospect of optimizing, and possibly even redesigning, our company's data architecture to support our next generation of products and data initiatives, we encourage you to apply and be part of our dynamic team shaping the future of our data operations!

Responsibilities

• Create and maintain optimal data pipeline architecture
• Assemble large complex data sets that meet functional / non-functional business requirements.
• Identify design and implement internal process improvements: automating manual processes optimising data delivery re-designing infrastructure for greater scalability etc.
• Build the infrastructure required for optimal extraction transformation and loading of data from a wide variety of data sources
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive Product Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure
• Create data tools for analytics and data scientist team members that assist them in building and optimising our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

Requirements

Mandatory skills:

• Data Processing: Apache Spark - Scala or Python
• Data Storage: Apache HDFS or respective cloud alternative
• Resource Manager: Apache Yarn or respective cloud alternative
• Lakehouse: Apache Hive/Kyuubi or alternative
• Workflow Scheduler: Airflow or alternative

Nice to have skills:

• Functional programming
• Apache Kafka
• Kubernetes
• Stream processing
• CI/CD