Lead Data Engineer

Job Description & Summary

We are seeking an experienced Lead Data Engineer with a profound understanding of banking data and extensive experience in data engineering. The ideal candidate will have hands-on experience with PySpark and Scala Spark, and significant prior experience working with the Quantexa platform. Familiarity with cloud environments is also essential.

Responsibilities:

• Lead the design, development, and maintenance of robust and scalable data processing pipelines using PySpark and Scala Spark;
• Collaborate with cross-functional teams to define, design, and implement new features;
• Cleanse, parse, and standardize global datasets;
• Perform data classification and entity extraction/resolution;
• Optimize and improve data processing performance through big data testing;
• Utilize pre-packaged code for processing and utilizing third-party data sources;
• Ensure data quality and integrity through efficient data manipulation;
• Deploy updates and fixes, build and maintain highly available systems;
• Implement and improve monitoring and alerting;
• Perform root cause analysis for production errors and investigate/resolve technical issues;
• Work in an Agile environment and contribute to the continuous improvement of our development processes;
• Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement.

Requirements:

• Bachelor’s degree in Computer Science, Engineering, or a related field;
• At least 6 years of hands-on experience in data engineering;
• Profound understanding of banking data and related processes;
• Solid experience with big data technologies like PySpark and Scala Spark;
• Hands-on experience with the Quantexa platform for more than 3 years is a must;
• Prior experience working with cloud environments;
• Familiarity with Agile development methodologies;
• Strong problem-solving skills and attention to detail;
• Excellent communication and teamwork abilities;
• Experience with provisioning software/frameworks like Elasticsearch, Spark, Hadoop, PostgreSQL;
• Knowledge of containerization technologies like Docker and orchestration tools like Kubernetes;
• Experience working with the Spring Boot Java framework.

We offer:

• Interesting career development options and challenging assignments for top Romanian and international clients;
• International mobility and short/medium-term projects abroad;
• A professional and team-oriented work environment;
• Great colleagues to work with;
• Up-to-date technologies and methodology;
• Professional development and training opportunities in Romania and abroad.