Data Engineer

We are looking for a motivated, independent, and well organised individual with experience of ETL processes using AWS services and multiple database technologies.

The role also involves task management, ensuring the sourcing of data, dealing with 3rd parties and follow through the full data life cycle.The data engineer will be responsible for the ETL processes as well as managing and supporting the full data cycle, from sourcing the data to ensuring its correct processing. The role will rely on high levels of automation and a strong attention to detail to making sure that our processes flow smoothly and are effective. The postholder will work with our Analytic team to ensure
the data is processed according to requirements.

Role Responsibilities

• Ownership of the full data lifecycle, from sourcing to processing to delivery.
• Establish an ETL process for new datasets including full validation and data quality assessment.
• Support, and operation of data processes.
• Work with the wider development team to improve the processes.
• Extraction of data for prospective customers to test models

Requirements

Skills & Experience

ESSENTIAL:

• 3-5 years’ experience in a data management role including experience of developing ETL pipelines ideally in AWS ecosystem
• Relational Database Management Systems (RDBMS) , SQL, NoSQL (DynamoDb, Redshift),
• Experience in dealing with 3rd party data vendor.
• Experience in defining data process improvement, defining requirements.
• General, overall knowledge of AWS Cloud services (e.g AWS Batch, Lambda, Step Functions ,S3/Event), AWS certified an advantage.
• Data engineering and ETL pipeline building,
• Python and or .net programming or similar and interfacing with RDBMS
• Experience with visualization tool (PowerBI/Tableau).
• Strong understanding of how to use automation to achieve processing scalability and speed.
• Experience working with data, including processes for data validation and quality control.
• University degree in information technology or similar

#LI-CI #Hybrid