Senior Data Engineer

What will you do?

• Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes, optimising data delivery, scabilibility of technology
• Implement enhancements and new features across data systems
• Improve streamline processes within data systems with support from Senior Data Engineer Test CI/CD process for optimal data pipelines
• Assemble large, complex data sets that meet functional / non functional business requirements
• Highly efficient in ETL processes
• Develop and conduct unit tests on data pipelines as well as ensuring data consistency
• Develop and maintain automated monitoring solutions
• Support reporting and analytics infrastructure
• Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems i.e. AWS/database security
• Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users
• Ensure best practice is implemented and maintained on our database in terms of collations, database engines, indices etc.

Requirements

• BSc degree in Computer Science/information systems/Engineering or related technical field or equivalent work experience
• 3+ years related work experience
• Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
• Experienced in Python
• Experienced in SQL (PostgreSQL, MS SQL)
• Experienced in using a cloud services: AWS, Azure or GCP
• Proficiency in version control, CI/CD and Github
• Understanding/experience in Glue and PySpark highly desirable
• Experience in managing data life cycle
• Proficiency in manipulating, processing and architecting large disconnected data sets for
• analytical requirements
• Ability to maintain and optimise on processes supporting data transformation, data structures, metadata, dependency and workload management.
• Good understanding of data management principles - data quality assurance and governance
• Strong analytical skills related to working with unstructured datasets
• Understanding of message queuing, stream processing, and highly scalable ‘big data’ data
• stores
• Strong attention to detail
• Good communication and interpersonal skills

Additional information

Why work for us?

• We are fully remote! Work from anywhere in the world within a 3 hour SAST timezone
• We have co-working spaces in Cape Town, Mauritius, Kenya & Nigeria
• A cutting-edge development stack is waiting for your code contribution
• You will be working with over 80+ highly skilled technical minds (and expanding)
• Backing of the international media powerhouse Ringier AG, which allows us to access
• world-class development resources, shared knowledge, and a wealth of international
• business insights
• Our teams work with a wide range of tech stacks
• We are committed to the future growth prospects of our employees by allowing autonomy,
• flexibility and an opportunity to master your respective skill