Data Engineer

Requirements

• Create and maintain optimal data pipeline architecture,
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources like web, mobile, apis
• Create data tools for analytics and data science teams.
• Deployment of sophisticated analytics programs, machine learning models and statistical methods.
• Identify ways to improve data reliability, efficiency and quality

What You Will Need

• 3+ years of experience in a Data Engineer role.
• Experience building and optimising big data pipelines, architectures and data sets.
• Build systems supporting data transformation, data structures, metadata, dependency and workload management.
• Working knowledge of message queuing, stream processing, and highly scalable data stores.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases.
• Experience with object-oriented/object function scripting languages: Python, Java, Scala, etc.
• Experience with AWS/GCP cloud services: EC2, EMR, RDS, Redshift, DataProc, CloudSql, Bigquery

Benefits