Data Engineer

Appen is on the lookout for a seasoned Senior Data Engineer to join our rapidly expanding team. As a Senior Data Engineer, you will leverage your deep technical expertise and leadership skills to drive the development of cutting-edge software solutions that will shape the future of AI and machine learning.

Key Responsibilities

• Design, build, and manage large scale data infrastructures using a variety of AWS technologies such as Amazon Redshift, AWS Glue, Amazon Athena, AWS Data Pipeline, Amazon Kinesis, Amazon EMR, and Amazon RDS.
• Ensure the performance, availability, and security of datasets across our organization, utilizing AWS's robust suite of tools for data management.
• Collaborate with data scientists, software engineers, product managers and other key stakeholders to develop data-driven solutions and models.
• Translate complex functional and technical requirements into detailed design proposals and implement them .
• Mentor junior and mid-level data engineers, fostering a culture of continuous learning and improvement within the team.
• Identify, troubleshoot, and resolve complex data-related issues.
• Champion best practices in data management, ensuring the cleanliness, integrity, and accessibility of our data.
• Optimize and fine-tune data queries and processes for performance. Evaluate and advise on technological components, such as software, hardware, and networking capabilities, for database management systems and infrastructure.
• Stay informed on the latest industry trends and technologies to ensure our data infrastructure is modern and robust.

Qualifications

• Bachelor's Degree in Computer Science, Information Systems, or a related field. A Master's Degree is preferred.
• 3 - 5 years of hands-on experience with AWS data engineering technologies, such as Amazon Redshift, AWS Glue, AWS Data Pipeline, Amazon Kinesis, Amazon RDS, and Apache Airflow.
• Demonstrated proficiency in SQL and NoSQL databases, ETL tools, and data pipeline workflows.
• Experience with Python, and/or Java.
• Deep understanding of data structures, data modeling, and software architecture.
• Experience with AI and machine learning technologies is highly desirable.
• Strong problem-solving skills and attention to detail.
• Self-motivated and able to work independently, with excellent organizational and multitasking skills.
• Exceptional communication skills, with the ability to explain complex data concepts to non-technical stakeholders.